365 Commits

Author SHA1 Message Date
Benoit Chesneau
f4ac8e1f1b test: pass action name to dirty client and stabilize after TTOU spam
- /unlimited and /limited handlers passed the data dict where the dirty
  client expected the action (method) name, surfacing as a 500 from
  getattr(self, action) on the dirty worker. Pass 'process' as the
  action so the call routes to DirtyApp.process(data).
- TestUnlimitedApps now bumps worker count via TTIN and polls both apps
  for readiness before each test. The preceding TTOU-spam test pins the
  worker count at the LimitedTask floor (2) and the arbiter takes a
  moment to rebind apps to the surviving workers; the previous tests
  raced that rebind and saw 'No workers available'.
2026-05-04 10:32:44 +02:00
Benoit Chesneau
54d38afddf test: unblock docker fixtures on macOS hosts
- per_app_allocation: move host port from 8001 to 28001. OrbStack reserves
  8001 on macOS for vcom-tunnel which makes 'Bind: port already allocated'
  the default failure mode.
- dirty_ttin_ttou: pin BASE_URL to 127.0.0.1 instead of 'localhost'. macOS
  resolves 'localhost' to ::1 first; Docker Desktop / OrbStack only forward
  host ports on IPv4 so the IPv6 attempt resets and the test fixture treats
  the service as unhealthy.
- dirty_ttin_ttou: add setproctitle to the test image. The TTIN/TTOU tests
  count workers via 'pgrep -f dirty-worker', which only matches once
  gunicorn's util._setproctitle has actually renamed the processes.
2026-05-04 09:47:38 +02:00
Benoit Chesneau
68843c8893
Merge pull request #3621 from benoitc/fix/asgi-preserve-content-length-on-head-and-304
fix: keep Content-Length on HEAD and 304 responses
2026-05-03 22:56:24 +02:00
Benoit Chesneau
41ec7527db fix: keep Content-Length on HEAD and 304 responses
RFC 9110 §6.4.2 forbids Content-Length only on 1xx and 204 responses.
HEAD MAY include the Content-Length the same GET would return, and 304
MAY include the Content-Length the unconditional response would carry.
WSGI preserves app-supplied Content-Length on those statuses; ASGI was
stripping it indiscriminately for any no-body response.

Split the predicate: _response_forbids_content_length() returns True
only for 1xx/204; _strip_body_framing_headers(headers, status) always
strips Transfer-Encoding (no body, no chunked terminator) and strips
Content-Length only when forbidden.
2026-05-03 22:32:28 +02:00
Benoit Chesneau
112d5353c1 fix: enforce proxy_allow_ips and tighten PROXY parsing in ASGI
Three findings against the ASGI PROXY protocol path:

- High: an untrusted peer could send a PROXY v1/v2 header and have the
  client address surfaced to the app. _setup_callback_parser now passes
  proxy_protocol='off' to the parser when the peer is not in
  proxy_allow_ips. _effective_peername adds a defensive re-check.
- Medium: PROXY v1 TCP4/TCP6 addresses were copied as strings without
  validation. Validate with socket.inet_pton, mirroring the WSGI parser.
- Medium: PROXY v2 quietly mapped non-STREAM (DGRAM) protocols to
  UDP4/UDP6. gunicorn is an HTTP server; reject non-STREAM with
  InvalidProxyHeader, mirroring the WSGI parser.
2026-05-03 22:28:48 +02:00
Benoit Chesneau
35c6a2abef test: parametrize smuggling regression across python and fast parsers
The previous test forced http_parser='python' to avoid a hard
dependency on gunicorn_h1c. Now run the same scenario under both
parser implementations so the smuggling guard is exercised on every
supported request-line/header path.
2026-05-03 21:18:15 +02:00
Benoit Chesneau
0a736ea4a2 fix: keep _body_receiver alive across the keepalive smuggling gate
The smuggling guard added in #3614 reads self._body_receiver after
_handle_http_request returns to refuse keepalive on a body that did
not finish framing.  But _handle_http_request's finally cleared the
receiver before returning, so the gate always saw None and let
keepalive proceed unconditionally.  Move the clear into the
connection loop's per-iteration cleanup (it already had one there
for the same purpose).

Adds an end-to-end regression test that pipelines a partial-body POST
followed by a smuggled GET and asserts the second request is not
served and the transport closes.
2026-05-03 21:03:49 +02:00
Benoit Chesneau
fe3655b9d3 refactor: split BodyReceiver._closed into transport vs body-wait
_closed now means the client transport has gone away. Body-wait timeouts
flip a separate _body_wait_expired flag. Both still surface as
http.disconnect to the app, but downstream code can now distinguish 'the
socket is dead' from 'the body never finished framing in time' without
guessing which path set the flag.
2026-05-03 20:42:55 +02:00
Benoit Chesneau
2191832b8d fix: drop body framing on HEAD/204/304 even when framework set it
RFC 9110 forbids a body for HEAD requests and for 1xx/204/304 status
codes. PR #3614 stopped gunicorn from auto-applying chunked encoding
in those cases, but if the application explicitly emitted a
Content-Length or Transfer-Encoding header (and possibly body bytes),
gunicorn still passed them through. Now strip both headers, force
plain framing, and discard any body the app emits.
2026-05-03 20:17:00 +02:00
Benoit Chesneau
d6443e5a6e test: poll for control socket re-creation after SIGHUP
The previous assertion ran immediately after a 2s sleep and raced
the arbiter's socket re-creation on slow runners (observed flake on
FreeBSD 14.2 / Python 3.13). Replace with the wait_for_socket helper
already used elsewhere in the file.
2026-05-03 19:13:40 +02:00
Benoit Chesneau
8e25cb2400 fix: tighten keepalive gate and scope finish_body byte cap
- ASGI keepalive gate now keys on receiver._complete only. _closed is
  overloaded across transport disconnect and receive timeout; treating
  either as 'message complete' would re-enable the smuggling vector
  the previous PR was meant to close.
- Parser.finish_body's 64 KiB byte cap now applies only when an explicit
  deadline is given. Default invocations (notably __next__, used by
  base_async / sync workers) regain the prior unbounded drain so a
  partial drain does not silently desync the next request.
2026-05-03 18:37:45 +02:00
Benoit Chesneau
6f9ed30d23 lint: use dict literal and hoist mock import 2026-05-03 18:23:45 +02:00
Benoit Chesneau
e90b1c2c1e fix: address six WSGI/ASGI parser and protocol findings
- WSGI fast parser now applies the same per-header policy as the Python
  parser (Expect, secure_scheme_headers, forwarded_allow_ips trust gate,
  forwarder_headers / header_map). Shared helpers extracted on Message.

- ASGI keepalive no longer resets the parser when the previous request
  body was not fully framed; the connection closes instead, preventing
  request smuggling on pipelined connections.

- BodyReceiver._wait_for_data timeout flips _closed and yields
  http.disconnect rather than synthesizing more_body=False. Timeout
  honors cfg.timeout.

- ASGI chunked encoding now skips HEAD, 204, and 304 (matches
  Response.is_chunked in the WSGI path) via a small helper.

- _setup_callback_parser passes proxy_protocol to PythonProtocol; auto
  falls back to the Python parser when proxy_protocol != off (the C
  parser does not implement PROXY framing). _effective_peername swaps
  the transport peer with the PROXY-supplied client address.

- Parser.finish_body accepts a deadline and a 64KiB byte cap; gthread
  passes a deadline and abandons keepalive on incomplete drain so a
  stalled client cannot tie up a worker thread.
2026-05-03 18:19:08 +02:00
Benoit Chesneau
98eac0b04e chore: require gunicorn_h1c >=0.6.5 and drop last python_only marker
gunicorn_h1c 0.6.5 ships the Content-Length list-form rejection
(h1c #8). The last python_only marker can now come off
rfc9112_smuggle_cl_list_form_01.
2026-04-20 07:29:47 +02:00
Benoit Chesneau
377e8f81f9 test: skip fast parser on PyPy (gunicorn_h1c C extension is CPython-only) 2026-04-19 23:48:18 +02:00
Benoit Chesneau
408b1f0517 chore: require gunicorn_h1c >=0.6.4 and drop python_only markers
gunicorn_h1c 0.6.4 ships the RFC 9110/9112 hardening added in h1c #4,
#6, and #7: control chars in header values, request-target form/method
pairing, and forbidden trailer field-names. All the corresponding
fixtures now pass against the C parser, so their python_only markers
are removed.

The CL list form fixture stays marked — the C parser does not yet
reject Content-Length: "5, 5".
2026-04-19 23:40:58 +02:00
Benoit Chesneau
73e64364ca test: codify rejection of Content-Length list form (RFC 9112 section 6.3) 2026-04-19 20:49:32 +02:00
Benoit Chesneau
8450ae0d10 test: codify body-framing cases (RFC 9110 section 8.6 and RFC 9112 section 6.1) 2026-04-19 20:32:23 +02:00
Benoit Chesneau
e223e302af test: codify field-syntax cases (RFC 9110 section 5.5 and 5.6.2) 2026-04-19 14:02:58 +02:00
Benoit Chesneau
2391901b40 test: codify chunked size/extension edge cases (RFC 9112 section 7.1) 2026-04-19 13:21:20 +02:00
Benoit Chesneau
826bfc7e88 test: add failing fixtures for control chars in header value 2026-04-19 12:05:00 +02:00
Benoit Chesneau
3b3752eb90 test: add failing fixtures for forbidden trailer fields 2026-04-19 11:38:05 +02:00
Benoit Chesneau
62252223e0 test: codify rejection of relative-reference request-target (RFC 9112 section 3.2) 2026-04-19 11:23:00 +02:00
Benoit Chesneau
e7fd6a104f test: add failing fixture for authority-form with non-CONNECT method 2026-04-19 11:09:29 +02:00
Benoit Chesneau
82d33d4c71 fix: reject asterisk-form request-target outside OPTIONS (RFC 9112 section 3.2.4)
The Python WSGI and ASGI parsers both accepted `GET *` and similar; RFC
9112 restricts asterisk-form to OPTIONS. Both now raise InvalidRequestLine.

The fast (C) parser in gunicorn_h1c does not yet enforce this, so the
fixture is marked python_only via a new sidecar flag honored by the WSGI
and ASGI invalid-request harnesses.
2026-04-19 10:43:01 +02:00
Benoit Chesneau
2c57071675 test: add failing fixture for asterisk-form with non-OPTIONS method 2026-04-19 10:37:14 +02:00
Benoit Chesneau
e896a653a4 test: codify absolute-form and IPv6 authority request-target vectors (phase 2A) 2026-04-19 10:15:39 +02:00
Benoit Chesneau
369b8d7d2c
Merge pull request #3591 from benoitc/test/rfc9112-compliance-corpus-phase1
test: codify RFC 9112 request-target and TE/CL vectors (phase 1)
2026-04-19 10:12:33 +02:00
Benoit Chesneau
f1c204626f test: codify RFC 9112 request-target and TE/CL vectors (phase 1)
Six treq fixtures covering gaps: absolute-form, asterisk-form (OPTIONS *),
authority-form (CONNECT), TE codings stacking (gzip/identity before chunked),
and the CL + TE:chunked smuggling vector.

Phase 1 of a staged corpus expansion; fixtures only, no parser changes.
2026-04-19 09:52:15 +02:00
ran
7ae6503dea fix: validate headers in early_hints callback to match process_headers
The early_hints callback constructs 103 Early Hints responses without
any header validation, while process_headers validates against TOKEN_RE
and HEADER_VALUE_RE for normal responses. This inconsistency means a
WSGI app passing unsanitized data to wsgi.early_hints could enable
HTTP response splitting via CRLF injection.

Apply the same TOKEN_RE/HEADER_VALUE_RE checks from process_headers to
the early_hints callback for defense-in-depth consistency.

Closes #3585
2026-04-13 17:21:24 +08:00
Benoit Chesneau
3936905c3f Fix RFC 9110 section reference: 8.6 not 15.2 2026-04-04 03:10:52 +02:00
Benoit Chesneau
97fcc6f1ee Update ASGI compatibility grid - 438/444 tests passing 2026-04-04 03:00:41 +02:00
Benoit Chesneau
06e59d252b Fix Litestar request handling - use raw ASGI receive for body/headers
Litestar internally caches request.body() and request.headers which
caused stale data to be returned on subsequent requests over keep-alive
connections. Access body via receive callable and headers directly from
scope to avoid this caching behavior.
2026-04-04 02:27:57 +02:00
Benoit Chesneau
db9030b7bc Fix Quart headers endpoint - normalize keys to lowercase
HTTP headers are case-insensitive. Normalize to lowercase for consistency
with tests and other frameworks.
2026-04-03 23:53:09 +02:00
Benoit Chesneau
746cc049d0 Skip HTTP 100 Continue test - invalid per RFC 7231
HTTP 100 Continue is an informational response that must be followed
by a final response. Testing it as a final response is invalid HTTP.
2026-04-03 23:13:27 +02:00
Benoit Chesneau
9c2bedceb7 Fix Litestar HTTP endpoints for compatibility tests
- Echo endpoint: add explicit status_code=200 (Litestar defaults to 201)
- Status endpoint: handle 204 No Content with empty body per HTTP spec
2026-04-03 23:13:22 +02:00
Benoit Chesneau
cbba5cb302 Fix Quart WebSocket close test app - add missing accept()
WebSocket connections must be accepted before they can be closed.
Added await websocket.accept() before await websocket.close(code).
2026-04-03 23:12:25 +02:00
Benoit Chesneau
65ba40b243 Update Docker setup to install gunicorn from local source
This allows testing local changes to gunicorn in the E2E test suite.
Previously containers were installing from GitHub master branch.

Also updates compatibility grid with latest test results (417/444, 93%).
2026-04-03 22:03:39 +02:00
Benoit Chesneau
cf92b2317a Fix duplicate Transfer-Encoding header for BlackSheep streaming
When frameworks like BlackSheep set Transfer-Encoding: chunked on
streaming responses, gunicorn was adding a second header without
checking if one already exists. This caused httpcore to reject the
response with "multiple Transfer-Encoding headers" error.

Fix checks for existing Transfer-Encoding header before adding one,
while still enabling chunked body encoding when the framework sets it.
2026-04-03 21:10:23 +02:00
Benoit Chesneau
3fc9a2f002 Fix WebSocket close handshake to comply with RFC 6455
- Add _close_sent, _close_received, _close_event state variables
- Server now waits for client's close frame response before marking
  connection as closed (5s timeout)
- Update _read_frames loop to continue reading after sending close
- Fix tests to simulate client close frame response
2026-04-03 14:53:36 +02:00
Benoit Chesneau
47bd20a7cb Fix HTTP 100 Continue adding Transfer-Encoding: chunked
Skip adding Transfer-Encoding: chunked for 1xx informational
responses per RFC 9110 Section 15.2.
2026-04-03 13:57:49 +02:00
Benoit Chesneau
136a124674 Add unit tests reproducing ASGI framework compatibility failures
Tests expose HTTP 100 Continue bug: gunicorn incorrectly adds
Transfer-Encoding: chunked to 1xx responses, violating RFC 9110.

Test results:
- 2 FAILED: HTTP 100 Continue (confirms bug)
- 20 PASSED: WebSocket close/binary/handshake

Coverage: websocket.py 62%, protocol.py 29%
2026-04-03 12:11:32 +02:00
Benoit Chesneau
26ae6e6f47 Add ASGI framework compatibility E2E test suite
Docker-based test suite validating gunicorn's ASGI worker against:
- Django + Channels
- FastAPI
- Starlette
- Quart
- Litestar
- BlackSheep

Tests cover HTTP scope, HTTP messages, WebSocket, lifespan protocol,
and streaming responses. Includes compatibility grid generator.

Results: 403/444 tests passed (90%)
2026-04-03 11:10:00 +02:00
Benoit Chesneau
1c82d4b518 Add ASGI test suite enhancement with 134 new tests
New test files covering areas identified as gaps compared to
Daphne and Uvicorn test coverage:

- test_asgi_header_security.py: Header validation, normalization,
  injection prevention
- test_asgi_error_handling.py: Application errors, body receiver
  errors, graceful shutdown
- test_asgi_protocol_http.py: HTTP connection management, chunked
  encoding, methods, scope building
- test_asgi_websocket_enhanced.py: WebSocket message limits,
  connection rejection, subprotocols
- test_asgi_lifespan.py: Lifespan message formats and behavior
- test_asgi_forwarded_headers.py: X-Forwarded-* and proxy header
  handling
2026-04-03 09:09:16 +02:00
Benoit Chesneau
4e9db71aeb
Merge pull request #3568 from benleembruggen/fix/h2-stream-ended-body-complete
fix: HTTP/2 ASGI body duplication in async_connection.py
2026-04-03 01:51:02 +02:00
Benoit Chesneau
7953c2585b Fix ASGI disconnect handling for Django-style apps
BodyReceiver.receive() now blocks after body is finished until actual
disconnect, instead of returning http.disconnect immediately. This fixes
Django's listen_for_disconnect task thinking client disconnected early.

Adds regression tests for the fix.

Fixes #3484
2026-04-02 23:55:27 +02:00
Ben Leembruggen
9243b2ad19 chore: remove integration test file and trim docstring 2026-04-01 11:57:03 +11:00
Ben Leembruggen
bcb13b1e74 Fix _handle_stream_ended to set _body_complete in async HTTP/2 handler
_handle_stream_ended() in async_connection.py (used by the ASGI worker)
did not set stream._body_complete = True or signal stream._body_event.
This caused the receive() closure in protocol.py to never see the body
as complete via the streaming path, so on the next call the fast path
re-read the entire body from BytesIO, doubling it.

The sync handler in connection.py already had a partial fix from #3559
but was also missing _body_event signalling, which is needed to unblock
any pending read_body_chunk() await.

Fixes https://github.com/benoitc/gunicorn/discussions/3567
2026-04-01 11:53:06 +11:00
Benoit Chesneau
3e2167c346
Add InvalidChunkExtension mapping and fast parser support for ASGI tests (#3565)
* Add InvalidChunkExtension to treq_asgi.py and fast parser support

- Add InvalidChunkExtension import and exception mapping for proper test
  coverage of bare CR rejection in chunk extensions per RFC 9112 7.1.1
- Add fast parser (H1CProtocol) support to treq_asgi.py and the ASGI
  invalid request tests
- Fast parser now receives limit configuration (limit_request_line,
  limit_request_fields, limit_request_field_size)
- Handle gunicorn_h1c's multiple ParseError classes from different modules
- Skip tests where fast parser has different validation than Python parser

* Handle gunicorn_h1c limit exceptions in ASGI protocol

Add handling for gunicorn_h1c.LimitRequestLine and
gunicorn_h1c.LimitRequestHeaders exceptions, matching the behavior
of the Python parser exceptions with appropriate HTTP status codes:
- LimitRequestLine: 414 URI Too Long
- LimitRequestHeaders: 431 Request Header Fields Too Large

* Refactor data_received to fix too-many-return-statements lint
2026-03-31 03:07:56 +02:00
Benoit Chesneau
8d08aaa2cb Fix --limit-request-line 0 to mean unlimited
Per documentation, limit_request_line=0 means unlimited. The code was
incorrectly treating 0 as "use default max" by checking <= 0 instead
of < 0.

For the fast C parser (gunicorn_h1c), which doesn't support 0 as
unlimited, pass a large value (1MB) instead. This applies to both
WSGI workers (http/message.py) and ASGI workers (asgi/protocol.py).

Fixes #3563
2026-03-26 23:42:14 +01:00