Replace RLock-based synchronization with a pipe-based method queue
for lock-free coordination between worker threads and main thread.
Key changes:
- Add PollableMethodQueue class using os.pipe() for wake-up signaling
- Non-blocking pipe (both ends) for BSD compatibility (FreeBSD, OpenBSD)
- Unified event loop using single poller.select() - no more futures.wait()
- Better graceful shutdown with connection draining within grace period
- Rename _keep to keepalived_conns, remove _lock entirely
- Add handle_exit() for SIGTERM, improve handle_quit() for SIGQUIT
- Add set_accept_enabled() for dynamic connection acceptance control
- Add wait_for_and_dispatch_events() with EINTR handling
Performance improvement: ~8% at high concurrency due to reduced
lock contention and non-blocking pipe operations.
Tests: 40 tests covering PollableMethodQueue, graceful shutdown,
keepalive management, error handling, and BSD compatibility.
Fixes#3146Closes#3157
This commit addresses three issues with the gthread worker:
1. Request body handling on keepalive
- Add finish_body() method to Parser to discard unread body bytes
- Call it before returning connections to the poller
- Prevents socket appearing readable due to leftover body
Fixes#3301
2. Timeout reliability with monotonic clock
- Replace time.time() with time.monotonic() in set_timeout()
- Replace time.time() with time.monotonic() in murder_keepalived()
- Prevents timeout issues caused by NTP adjustments
3. SSL error handling
- Move conn.init() from enqueue_req() to handle()
- SSL handshake now runs in worker thread, not main thread
- ENOTCONN errors during ssl_wrap_socket are caught per-connection
- Prevents entire worker crashes on SSL handshake failures
Also adds comprehensive unit tests for the gthread worker.
Closes#3303Closes#3308
Strip whitespace also *after* header field value.
Simply refuse obsolete header folding (a default-off
option to revert is temporarily provided).
While we are at it, explicitly handle recently
introduced http error classes with intended status code.
changes:
- Just follow the new TE specification (https://datatracker.ietf.org/doc/html/rfc9112#name-transfer-encoding)
here and accept to introduce a breaking change.
- gandle multiple TE on one line
** breaking changes ** : invalid headers and position will now return
an error.
New parser rule: refuse HTTP requests where a header field value
contains characters that
a) should never appear there in the first place,
b) might have lead to incorrect treatment in a proxy in front, and
c) might lead to unintended behaviour in applications.
From RFC 9110 section 5.5:
"Field values containing CR, LF, or NUL characters are invalid and
dangerous, due to the varying ways that implementations might parse
and interpret those characters; a recipient of CR, LF, or NUL within
a field value MUST either reject the message or replace each of those
characters with SP before further processing or forwarding of that
message."
chunk extensions are silently ignored before and after this change;
its just the whitespace handling for the case without extensions that matters
applying same strip(WS)->rstrip(BWS) replacement as already done in related cases
half-way fix: could probably reject all BWS cases, rejecting only misplaced ones
Note: This is unrelated to a reverse proxy potentially talking HTTP/3 to clients.
This is about the HTTP protocol version spoken to Gunicorn, which is HTTP/1.0 or HTTP/1.1.
Little legitimate need for processing HTTP 1 requests with ambiguous version numbers.
Broadly refuse.
Co-authored-by: Ben Kallus <benjamin.p.kallus.gr@dartmouth.edu>
Do the validation on the original, not the result from unicode case folding.
Background:
latin-1 0xDF is traditionally uppercased 0x53+0x53 which puts it back in ASCII
If we promise wsgi.input_terminated, we better get it right - or not at all.
* chunked encoding on HTTP <= 1.1
* chunked not last transfer coding
* multiple chinked codings
* any unknown codings (yes, this too! because we do not detect unusual syntax that is still chunked)
* empty coding (plausibly harmless, but not see in real life anyway - refused, for the moment)
Ambiguous mappings open a bottomless pit of "what is user input and what is proxy input" confusion.
Default to what everyone else has been doing for years now, silently drop.
see also https://nginx.org/r/underscores_in_headers
- Unify HEADER_RE and METH_RE
- Replace CRLF with SP during obs-fold processing (See RFC 9112 Section 5.2, last paragraph)
- Stop stripping header names.
- Remove HTAB in OWS in header values that use obs-fold (See RFC 9112 Section 5.2, last paragraph)
- Use fullmatch instead of search, which has problems with empty strings. (See GHSA-68xg-gqqm-vgj8)
- Split proxy protocol line on space only. (See proxy protocol Section 2.1, bullet 3)
- Use fullmatch for method and version (Thank you to Paul Dorn for noticing this.)
- Replace calls to str.strip() with str.strip(' \t')
- Split request line on SP only.
Co-authored-by: Paul Dorn <pajod@users.noreply.github.com>
This change defaults SSLContext to Python's ssl.create_default_context() and
marks ssl_version option as deprecated. The option value will be ignored and
warnign will be printed in stderr.
The ssl_version option was depending on old method of setting TLS min/max
version, which has not worked well anymore with modern Python versions.
Fixes#2223.
Unfortunately, eventlet doesn't implement GreenSocket.sendfile, so we have to do it for it.
Add gevent and eventlet to tox.ini and add tests to make sure we can at least import the workers. Some tests that this actually functions would be nice...
Update the gevent and eventlet setup extras to require the versions that are enforced in their worker modules.
* load application from factory function
Use `ast.parse` to validate that the string passed to the CLI is either
an attribute name or a function call. Use `ast.literal_eval` to parse
any positional and keyword arguments to the function. Call the function
to get the real application.
Co-authored-by: Connor Brinton <connor.brinton@gmail.com>
* test coverage for util.import_app
* document app factory pattern
pytest.raises() returns exception info not the exception itself. They
changed implementation of exception info, so now .value property must be
used to get the exception instance and have proper output from str()
method.
https://github.com/pytest-dev/pytest/issues/5412
Signed-off-by: Martin Bašti <mbasti@redhat.com>
According RFC-7617 (inherited from RFC-2978) schema and parameter names are handled
case insensitively:
```
Note that both scheme and parameter names are matched case-
insensitively.
```
Signed-off-by: Martin Bašti <mbasti@redhat.com>
