BodyReceiver.receive() now blocks after body is finished until actual
disconnect, instead of returning http.disconnect immediately. This fixes
Django's listen_for_disconnect task thinking client disconnected early.
Adds regression tests for the fix.
Fixes#3484
_handle_stream_ended() in async_connection.py (used by the ASGI worker)
did not set stream._body_complete = True or signal stream._body_event.
This caused the receive() closure in protocol.py to never see the body
as complete via the streaming path, so on the next call the fast path
re-read the entire body from BytesIO, doubling it.
The sync handler in connection.py already had a partial fix from #3559
but was also missing _body_event signalling, which is needed to unblock
any pending read_body_chunk() await.
Fixes https://github.com/benoitc/gunicorn/discussions/3567
* Add InvalidChunkExtension to treq_asgi.py and fast parser support
- Add InvalidChunkExtension import and exception mapping for proper test
coverage of bare CR rejection in chunk extensions per RFC 9112 7.1.1
- Add fast parser (H1CProtocol) support to treq_asgi.py and the ASGI
invalid request tests
- Fast parser now receives limit configuration (limit_request_line,
limit_request_fields, limit_request_field_size)
- Handle gunicorn_h1c's multiple ParseError classes from different modules
- Skip tests where fast parser has different validation than Python parser
* Handle gunicorn_h1c limit exceptions in ASGI protocol
Add handling for gunicorn_h1c.LimitRequestLine and
gunicorn_h1c.LimitRequestHeaders exceptions, matching the behavior
of the Python parser exceptions with appropriate HTTP status codes:
- LimitRequestLine: 414 URI Too Long
- LimitRequestHeaders: 431 Request Header Fields Too Large
* Refactor data_received to fix too-many-return-statements lint
Per documentation, limit_request_line=0 means unlimited. The code was
incorrectly treating 0 as "use default max" by checking <= 0 instead
of < 0.
For the fast C parser (gunicorn_h1c), which doesn't support 0 as
unlimited, pass a large value (1MB) instead. This applies to both
WSGI workers (http/message.py) and ASGI workers (asgi/protocol.py).
Fixes#3563
AsyncRequest was the legacy pull-based async HTTP parser, now replaced
by the push-based CallbackRequest/PythonProtocol. Remove the unused
code and associated tests.
Update to gunicorn_h1c >= 0.6.3 which adds InvalidChunkExtension
validation for rejecting chunk extensions with bare CR bytes per
RFC 9112.
Changes:
- Update pyproject.toml to require gunicorn_h1c >= 0.6.3
- Add InvalidChunkExtension exception to gunicorn/asgi/parser.py
- Handle InvalidChunkExtension from both Python and C parsers in protocol.py
- Add chunk extension validation tests
- Update treq.py badrequest class to support hex escapes
Handle chunked encoding edge case where connection closes before
final CRLF after zero-chunk. Skip WSGI-specific tests (casefold,
underscore headers) that don't apply to ASGI.
receive_data() stores every DATA frame in both _body_chunks (list)
and request_body (BytesIO). The receive() closure in
_handle_http2_request() has two read paths: a streaming path that
pops from _body_chunks, and a fast path that reads from BytesIO.
After the streaming path consumed the body, the fast path could
re-read the same data from BytesIO because body_received was never
set in the streaming return path. This caused the application to
receive a doubled request body (e.g. 18 bytes sent, 36 bytes
received), breaking JSON parsing with "Extra data" errors.
Fix: set body_received = True in the streaming path when
_body_complete is True, preventing the fast path from re-reading
already-consumed data.
Fixes#3558
* fix: add __iter__ and __next__ to FileWrapper for PEP 3333 compliance
The WSGI spec (PEP 3333) requires that wsgi.file_wrapper return an
iterable object. Gunicorn's FileWrapper only implemented __getitem__,
which technically makes it iterable via old-style iteration but breaks
code that explicitly relies on the iterator protocol (e.g., calling
iter() or using next()).
This adds __iter__ (returning self) and __next__ to make FileWrapper
a proper iterator, maintaining backward compatibility with existing
__getitem__-based usage.
Fixes#3396
* Fix lint: move imports to top of file
---------
Co-authored-by: contributor <noreply@users.noreply.github.com>
Co-authored-by: Benoit Chesneau <bchesneau@gmail.com>
The previous default /run/gunicorn.ctl requires root permissions.
Now uses $XDG_RUNTIME_DIR/gunicorn.ctl if available, otherwise
$HOME/.gunicorn/gunicorn.ctl. This works on Linux, FreeBSD, OpenBSD,
and macOS without requiring elevated privileges.
- Add _get_default_control_socket() helper in config.py
- Create parent directory automatically with 0o700 permissions
- Update gunicornc CLI to use the same default path
- Add unit tests for path selection and directory creation
- Fix body receiver timeout handling to prevent infinite loops
- Add WebSocket data forwarding via callbacks instead of StreamReader
- Fix HTTP/2 stream race condition where DATA frames arrive before first read
- Update WebSocketProtocol constructor (removed reader parameter)
Add double-check after clearing _data_event to prevent deadlock when
data arrives between clear() and wait(). The race condition occurred
when:
1. Task A checks buffer, needs more data
2. Task A clears _data_event
3. Task B (feed_data) sets event
4. Task A awaits on cleared event - deadlock
The fix re-checks the buffer after clear() to catch data that arrived
in the race window.
Also adds tests for edge cases: race condition simulation, EOF during
wait, fragmented message reassembly, and control frames during
fragmentation.
- LimitRequestLine now accepts optional max_size parameter
- Use default max limits when limit_request_line or limit_request_field_size is 0
- Add tests validating default max enforcement (8190 bytes)
- Handle alternate exceptions from fast parser in test_invalid_requests
Require gunicorn_h1c >= 0.4.1 for fast parser mode. Add new exception
types and limit parameters to PythonProtocol for parity with C parser.
Update tests to parametrize across both parser implementations.
Add test suite that exercises both PythonProtocol and H1CProtocol
implementations with identical test cases using pytest parametrization.
Tests cover request line parsing, headers, body handling (Content-Length
and chunked), connection handling, parser reset, and callback behavior.
Add PythonProtocol class that mirrors H1CProtocol callback interface:
- Callbacks: on_message_begin, on_url, on_header, on_headers_complete,
on_body, on_message_complete
- Properties: method, path, http_version, headers, content_length,
is_chunked, should_keep_alive
- Methods: feed(data), reset()
- Supports Content-Length and chunked transfer encoding
Add CallbackRequest adapter for building requests from parser state.
Works with both H1CProtocol (C extension) and PythonProtocol.
Add unit tests for PythonProtocol and CallbackRequest.
- Replace datetime.now() with time.monotonic() for request timing
- Add access_log_enabled property to skip log work when disabled
- Rewrite BodyReceiver with Future-based waiting (no create_task)
- Remove StreamReader for HTTP/1.1, use direct bytearray buffering
- Add BufferReader wrapper for FastAsyncRequest compatibility
- Use pre-cached chunk prefixes in _send_body()
- Convert async methods to sync where no await needed
- Batch response writes (headers + body in single write)
Performance: 4,200 -> 69,500 req/s
Wire HttpParser to ASGI hot path, replacing AsyncRequest.parse() with
direct buffer-based parsing. Add FastAsyncRequest wrapper for body
reading. Replace per-request Queue/Task with BodyReceiver for on-demand
body reading. Keep headers as bytes end-to-end to avoid conversion
overhead. Add backpressure control and keepalive timer. Cache response
status lines and Date header.
Benchmark shows 3x improvement: ~875K req/s for simple GET (was ~340K).
- Integrate gunicorn_h1c fast parser into WSGI Request class
- Add _check_fast_parser() and _parse_fast() methods
- Tests use Python parser for consistent validation behavior
- Update config description to reflect all worker types
- Use os.register_at_fork() to properly handle fork() with asyncio
- Start control server after initial workers spawn, not before
- Change default socket path to /run/gunicorn.ctl (like BIRD)
- Add integration tests for sync, gthread, and gevent workers
Fixes#3509
Add a timeout when waiting for initial request data in worker threads.
If no data arrives within 5 seconds, the connection is deferred back
to the main poller instead of blocking the thread indefinitely.
This fixes a regression from v24 where connections were submitted
directly to the thread pool after accept(). In v23, connections were
registered with the poller first and only submitted when data arrived.
The new hybrid approach maintains the performance benefits for fast
clients (immediate processing) while protecting against slow-client
scenarios that can exhaust the thread pool and cause health check
timeouts.
Changes:
- Add _DEFER sentinel and DEFAULT_WORKER_DATA_TIMEOUT constant
- Add TConn.wait_for_data() method using selectors
- Add TConn.data_ready flag to track data availability
- Add pending_conns deque for deferred connections
- Add on_pending_socket_readable() callback
- Add murder_pending() to clean up timed-out pending connections
- Modify handle() to wait for data with timeout before processing
- Modify finish_request() to handle _DEFER and register with poller
Fixes#3518
- Read chunk size lines and trailers in 64-byte blocks instead of 1 byte
at a time, pushing back excess data to the unreader buffer
- Reuse BytesIO buffers with truncate/seek instead of creating new
objects to reduce GC pressure in AsyncUnreader
- Use bytearray.find() directly instead of converting to bytes first
in header parsing loop
- Use index-based iteration for header parsing instead of list.pop(0)
which is O(n) per pop vs O(1) for index access
Add tests for the optimized parsing code paths.
- Use preexec_fn=os.setsid to create new process group
- Send signals to process group with os.killpg() instead of single process
- Add explicit timeout and graceful-timeout to gunicorn command
- Fixes test failures on PyPy 3.10 where signals weren't propagating properly
Replace signal-based dirty add/remove with protocol messages:
- Add MSG_TYPE_MANAGE to dirty protocol for worker management
- Add MANAGE_OP_ADD and MANAGE_OP_REMOVE operation codes
- Add handle_manage_request() in DirtyArbiter
- Update handlers to send messages instead of SIGTTIN/SIGTTOU signals
New workers only load apps that haven't reached their worker limits.
When all apps are at their limits, returns reason in response.
Only increment num_workers when a worker is actually spawned.
Add MSG_TYPE_STATUS to dirty protocol to allow querying the dirty
arbiter for its workers. The control socket now connects to the
dirty arbiter socket to retrieve worker information.
Add a control socket server and CLI client for runtime management
of Gunicorn instances, similar to birdc for BIRD routing daemon.
Features:
- Control socket server running in arbiter process (asyncio/threaded)
- gunicornc CLI with interactive and single-command modes
- JSON protocol with length-prefixed framing
- Commands: show workers/stats/config/listeners/dirty, worker add/remove/kill,
dirty add/remove, reload, reopen, shutdown
- Stats tracking (uptime, workers spawned/killed, reloads)
- Configurable socket path and permissions
New config options:
- control_socket: Unix socket path (default: gunicorn.ctl)
- control_socket_mode: Socket permissions (default: 0o600)
- --no-control-socket: Disable control socket
Add support for SIGTTIN and SIGTTOU signals to the dirty arbiter,
allowing dynamic scaling of dirty workers at runtime without restarting
gunicorn.
Changes:
- Add TTIN/TTOU to DirtyArbiter.SIGNALS
- Add num_workers instance variable for dynamic count
- Add _get_minimum_workers() to enforce app worker constraints
- Add signal handlers for TTIN (increase) and TTOU (decrease)
- Update manage_workers() to use dynamic count
- Add documentation for dynamic scaling
- Add unit tests for signal handling
- Add Docker integration tests
The minimum worker constraint ensures TTOU cannot reduce workers below
what apps require (e.g., if an app has workers=3, minimum is 3).
Closes#3489
* feat(dirty): add stash - global shared state between workers
Add a simple key-value store (stash) that allows dirty workers to share
state through the arbiter. Tables are stored directly in arbiter memory
for fast access and simplicity.
Features:
- Auto-create tables on first access
- Dict-like interface via stash.table()
- Pattern matching for keys (glob patterns)
- Module-level API: stash.put(), stash.get(), stash.delete(), etc.
Usage:
from gunicorn.dirty import stash
stash.put("sessions", "user:1", {"name": "Alice"})
user = stash.get("sessions", "user:1")
# Or dict-like
sessions = stash.table("sessions")
sessions["user:1"] = {"name": "Alice"}
New files:
- gunicorn/dirty/stash.py - Client API and StashTable class
- Protocol additions for MSG_TYPE_STASH and STASH_OP_* codes
Note: Tables are ephemeral - lost if arbiter restarts.
* test(dirty): add tests for stash protocol and encoding
Test coverage for:
- Stash message creation and encoding
- Protocol constants (MSG_TYPE_STASH, STASH_OP_*)
- Error classes (StashError, StashTableNotFoundError, StashKeyNotFoundError)
- StashTable dict-like interface
- Edge cases: unicode, complex values, special patterns
* example(dirty): add stash usage example and integration tests
- Add SessionApp to dirty_app.py demonstrating stash usage
- Add /session/* endpoints to wsgi_app.py
- Add test_stash_integration.py with Docker tests
- Update docker-compose.yml with stash-test service
- Fix: Set GUNICORN_DIRTY_SOCKET in dirty arbiter for worker access
* docs(dirty): add stash documentation
JSON serializes all dict keys as strings, so for compatibility the TLV
encoder should do the same. This fixes an error when tasks return dicts
with integer keys (e.g., aggregation results grouped by numeric ID).
Update client and streaming tests to work with the binary protocol:
- Update MockStreamWriter/MockStreamReader to use BinaryProtocol
- Replace string request IDs with integers
- Update test assertions to decode binary protocol messages
- Use HEADER_SIZE and decode_header/decode_message instead of old API
