3478 Commits

Author SHA1 Message Date
Benoit Chesneau
ba1aaa5e33 Fix non-ASCII URL handling in ASGI worker
Percent-decode path to UTF-8 and preserve raw_path as original bytes
per ASGI spec. Fixes #3543
2026-03-22 16:35:28 +01:00
Benoit Chesneau
932331d8a4 Regenerate settings.md 2026-03-22 16:23:46 +01:00
Benoit Chesneau
4ce6aa1f3e Fix limit handling and add default max limit tests
- LimitRequestLine now accepts optional max_size parameter
- Use default max limits when limit_request_line or limit_request_field_size is 0
- Add tests validating default max enforcement (8190 bytes)
- Handle alternate exceptions from fast parser in test_invalid_requests
2026-03-22 16:17:55 +01:00
Benoit Chesneau
2bf2632944 Update docs and changelog for gunicorn_h1c 0.4.1 integration 2026-03-22 13:44:31 +01:00
Benoit Chesneau
03cc85ef48 Integrate gunicorn_h1c 0.4.1 exception types and limit parameters
Require gunicorn_h1c >= 0.4.1 for fast parser mode. Add new exception
types and limit parameters to PythonProtocol for parity with C parser.
Update tests to parametrize across both parser implementations.
2026-03-22 13:43:18 +01:00
Benoit Chesneau
f308e7abfa Add parametrized tests for ASGI callback parsers
Add test suite that exercises both PythonProtocol and H1CProtocol
implementations with identical test cases using pytest parametrization.
Tests cover request line parsing, headers, body handling (Content-Length
and chunked), connection handling, parser reset, and callback behavior.
2026-03-22 09:23:47 +01:00
Benoit Chesneau
86c0baf933 Fix lint issues in ASGI parser and protocol 2026-03-22 09:00:48 +01:00
Benoit Chesneau
8ad49b8df3 Update settings.md for simplified http_parser options 2026-03-22 08:09:35 +01:00
Benoit Chesneau
e8150e27b2 Simplify ASGI parser to always use callback mode
Remove pull-based HttpParser path and always use callback-based parsing:

- Remove HttpParser, ParseResult, FastAsyncRequest classes from parser.py
- Remove BufferReader, _handle_connection_fast(), _parse_request_fast()
- Update _setup_callback_parser() to handle auto/fast/python modes
- Fix race condition when data arrives before _handle_connection starts
- Simplify http_parser config to auto/fast/python (remove callback modes)

Parser selection for ASGI:
- auto: H1CProtocol if available, else PythonProtocol
- fast: H1CProtocol required (error if unavailable)
- python: PythonProtocol only

Reduces code by ~1150 lines while maintaining performance.
2026-03-22 02:02:03 +01:00
Benoit Chesneau
87bfb7d190 Add RFC 7230 validation for chunked transfer-encoding
Validate after fast parser returns:
- Reject chunked with HTTP/1.0
- Reject chunked + Content-Length conflict
2026-03-22 00:18:25 +01:00
Benoit Chesneau
0ca0d0cb02 Fix body polling and HTTP/2 request streaming
- Replace 100ms polling with event-based waiting in BodyReceiver
- Stream HTTP/2 request bodies instead of buffering entire uploads
- Add timeout handling for disconnect detection
2026-03-22 00:01:17 +01:00
Benoit Chesneau
ea37eaaa6d Add streaming body support to HTTP2Stream
- Add _body_chunks, _body_event, _body_complete fields for streaming
- Modify receive_data() to populate chunks queue alongside BytesIO
- Add async read_body_chunk() method for streaming body reads

This enables HTTP/2 request body streaming instead of buffering
entire uploads, reducing memory usage for large file uploads.
2026-03-22 00:00:37 +01:00
Benoit Chesneau
464cbbfad5 Add write flow control and HTTP/2 streaming
- Add FlowControl class for transport-level write backpressure
- Integrate flow control into HTTP/1.1 protocol to prevent memory
  issues with large streaming responses
- Set write buffer high water mark to 64KB
- Add pause_writing/resume_writing protocol callbacks
- Stream HTTP/2 responses immediately instead of buffering
- Add _convert_h2_headers helper for cleaner header conversion
2026-03-21 23:50:06 +01:00
Benoit Chesneau
22bdca22e1 Integrate callback parsers into ASGI protocol
Add callback parser support to ASGIProtocol:
- Add _handle_connection_callback() for callback-based parsing
- Add parser callbacks: _on_headers_complete, _on_body, _on_message_complete
- Update data_received() to feed callback parser
- Add _setup_callback_parser() with H1CProtocol/PythonProtocol selection

Add http_parser config options:
- callback: Use callback parser (H1CProtocol if available, else PythonProtocol)
- fast-callback: Require H1CProtocol callback parser

Callback parsing moves HTTP parsing to data_received(), reducing async
overhead in the request handling loop.
2026-03-21 23:24:49 +01:00
Benoit Chesneau
23c7210b67 Bump gunicorn_h1c to 0.2.0 for callback parser 2026-03-21 23:24:38 +01:00
Benoit Chesneau
ae7653057f Add callback-based HTTP parser for ASGI protocol
Add PythonProtocol class that mirrors H1CProtocol callback interface:
- Callbacks: on_message_begin, on_url, on_header, on_headers_complete,
  on_body, on_message_complete
- Properties: method, path, http_version, headers, content_length,
  is_chunked, should_keep_alive
- Methods: feed(data), reset()
- Supports Content-Length and chunked transfer encoding

Add CallbackRequest adapter for building requests from parser state.
Works with both H1CProtocol (C extension) and PythonProtocol.

Add unit tests for PythonProtocol and CallbackRequest.
2026-03-21 23:24:23 +01:00
Benoit Chesneau
7818401182 Optimize ASGI protocol for 16x performance improvement
- Replace datetime.now() with time.monotonic() for request timing
- Add access_log_enabled property to skip log work when disabled
- Rewrite BodyReceiver with Future-based waiting (no create_task)
- Remove StreamReader for HTTP/1.1, use direct bytearray buffering
- Add BufferReader wrapper for FastAsyncRequest compatibility
- Use pre-cached chunk prefixes in _send_body()
- Convert async methods to sync where no await needed
- Batch response writes (headers + body in single write)

Performance: 4,200 -> 69,500 req/s
2026-03-21 22:20:05 +01:00
Benoit Chesneau
fa967743c0 Optimize ASGI performance with fast parser integration
Wire HttpParser to ASGI hot path, replacing AsyncRequest.parse() with
direct buffer-based parsing. Add FastAsyncRequest wrapper for body
reading. Replace per-request Queue/Task with BodyReceiver for on-demand
body reading. Keep headers as bytes end-to-end to avoid conversion
overhead. Add backpressure control and keepalive timer. Cache response
status lines and Date header.

Benchmark shows 3x improvement: ~875K req/s for simple GET (was ~340K).
2026-03-21 11:36:46 +01:00
Benoit Chesneau
d89564b83c Add HTTP parser benchmark comparing Python vs Fast
Benchmarks WSGI and ASGI parsers with:
- Simple GET request (35 bytes)
- Medium POST request (192 bytes, 7 headers)
- Complex POST request (891 bytes, 18 headers)

Results show fast parser (gunicorn_h1c) is:
- WSGI: ~1.9x faster than Python parser
- ASGI: ~2.7x faster than Python parser
2026-03-21 09:33:14 +01:00
Benoit Chesneau
7f175fb171 Add fast HTTP parser support for WSGI workers
- Integrate gunicorn_h1c fast parser into WSGI Request class
- Add _check_fast_parser() and _parse_fast() methods
- Tests use Python parser for consistent validation behavior
- Update config description to reflect all worker types
2026-03-21 09:29:01 +01:00
Benoit Chesneau
b833a9b6df Add optional fast HTTP parser for ASGI workers
- Add http_parser config setting (auto/fast/python)
- Add gunicorn_h1c as optional dependency [fast]
- Add unified HttpParser class with fallback to pure Python
- Parser tries gunicorn_h1c in 'auto' mode, falls back gracefully
- 'fast' mode requires gunicorn_h1c, 'python' forces pure Python

Install with: pip install gunicorn[fast]
2026-03-21 09:19:41 +01:00
Benoit Chesneau
2cc38503b7
Merge pull request #3514 from lukqw/enrich-error-logging
chore(logging): enrich request handling log line with request method
2026-03-13 10:25:38 +01:00
Benoit Chesneau
8caf79ec64
Merge pull request #3494 from Juneezee/docs/forwarded_allow_ips
docs: update forwarded_allow_ips to Markdown
2026-03-09 13:35:57 +01:00
Benoit Chesneau
667f76c59f
Merge pull request #3533 from benoitc/dependabot/github_actions/actions/upload-artifact-7
chore(deps): bump actions/upload-artifact from 6 to 7
2026-03-07 00:00:31 +01:00
Benoit Chesneau
24e7ced609
Merge pull request #3520 from benoitc/fix/control-socket-issues
fix(ctl): prevent fork deadlock and file watcher triggers
2026-03-06 23:58:57 +01:00
dependabot[bot]
7d101cbb40
chore(deps): bump actions/upload-artifact from 6 to 7
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 6 to 7.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-01 21:42:25 +00:00
Benoit Chesneau
7499da9a17 docs: regenerate settings.md for control_socket default 2026-02-27 15:16:40 +01:00
Benoit Chesneau
474f3ffa1e fix(lint): resolve pylint warnings in ctl/server and gtornado
- Replace global statement with module-level dict in ctl/server.py
- Fix argument rename warning in gtornado.py nested class method
2026-02-27 02:23:08 +01:00
Benoit Chesneau
089ad45818 fix(ctl): prevent fork deadlock in control socket server
- Use os.register_at_fork() to properly handle fork() with asyncio
- Start control server after initial workers spawn, not before
- Change default socket path to /run/gunicorn.ctl (like BIRD)
- Add integration tests for sync, gthread, and gevent workers

Fixes #3509
2026-02-26 20:54:35 +01:00
Benoit Chesneau
d606336de4
Merge pull request #3523 from bysiber/fix-pidfile-fd-leak
fix: ensure pidfile fd is closed on rename failure
2026-02-20 13:43:14 +01:00
Benoit Chesneau
32e46a58ce
Merge pull request #3522 from bysiber/fix-sd-notify-unbound-error
fix: prevent UnboundLocalError in sd_notify when socket creation fails
2026-02-20 13:42:51 +01:00
Benoit Chesneau
cef6b337d8
Merge pull request #3519 from benoitc/fix/gthread-slow-client-resilience
fix(gthread): prevent thread pool exhaustion from slow clients
2026-02-20 10:18:08 +01:00
Benoit Chesneau
eb6b69377c
Merge pull request #3513 from benoitc/feature/asgi-performance-optimizations
perf(asgi): optimize HTTP parser for improved performance
2026-02-20 10:17:38 +01:00
Kadir Can Ozden
6ada1ce03a fix: ensure pidfile fd is closed on rename failure 2026-02-20 06:17:29 +03:00
Kadir Can Ozden
41c6bf8e3e fix: prevent UnboundLocalError in sd_notify when socket creation fails 2026-02-20 06:16:54 +03:00
Benoit Chesneau
d3f80e8cfd fix(ctl): prevent fork deadlock and file watcher triggers
Fix two issues with the control socket feature introduced in 25.1.0:

1. Fork deadlock (#3509): The control server runs asyncio in a background
   thread. When fork() is called to spawn workers, locks held by asyncio
   can remain locked in the child process, causing deadlocks. Fix by
   stopping the control server before fork and restarting after fork in
   the parent.

2. File watcher triggers (#3508): The default socket path `gunicorn.ctl`
   was created in the app directory, triggering file watchers in dev
   servers (e.g., GAE dev_appserver). Fix by changing the default path
   to `/tmp/gunicorn.ctl`.

Fixes #3508
Fixes #3509
2026-02-19 15:57:41 +01:00
Benoit Chesneau
b5f127e99b fix(gthread): prevent thread pool exhaustion from slow clients
Add a timeout when waiting for initial request data in worker threads.
If no data arrives within 5 seconds, the connection is deferred back
to the main poller instead of blocking the thread indefinitely.

This fixes a regression from v24 where connections were submitted
directly to the thread pool after accept(). In v23, connections were
registered with the poller first and only submitted when data arrived.

The new hybrid approach maintains the performance benefits for fast
clients (immediate processing) while protecting against slow-client
scenarios that can exhaust the thread pool and cause health check
timeouts.

Changes:
- Add _DEFER sentinel and DEFAULT_WORKER_DATA_TIMEOUT constant
- Add TConn.wait_for_data() method using selectors
- Add TConn.data_ready flag to track data availability
- Add pending_conns deque for deferred connections
- Add on_pending_socket_readable() callback
- Add murder_pending() to clean up timed-out pending connections
- Modify handle() to wait for data with timeout before processing
- Modify finish_request() to handle _DEFER and register with poller

Fixes #3518
2026-02-19 15:21:01 +01:00
Benoit Chesneau
3f0c1f9ce5 fix(lint): remove unused io import 2026-02-19 01:02:14 +01:00
lukqw
aa29987be5 chore(logging): enrich request handling log line with request method 2026-02-18 10:12:14 +01:00
Benoit Chesneau
b3b98b1322 perf(asgi): optimize HTTP parser for improved performance
- Read chunk size lines and trailers in 64-byte blocks instead of 1 byte
  at a time, pushing back excess data to the unreader buffer
- Reuse BytesIO buffers with truncate/seek instead of creating new
  objects to reduce GC pressure in AsyncUnreader
- Use bytearray.find() directly instead of converting to bytes first
  in header parsing loop
- Use index-based iteration for header parsing instead of list.pop(0)
  which is O(n) per pop vs O(1) for index access

Add tests for the optimized parsing code paths.
2026-02-18 10:00:46 +01:00
Benoit Chesneau
2d4310116d docs: merge gunicornc into 25.1.0 release 2026-02-13 12:03:48 +01:00
Benoit Chesneau
bf4ad8d610 docs: update 25.1.0 release date to 2026-02-13 2026-02-13 11:59:42 +01:00
Benoit Chesneau
730350eb9f
Merge pull request #3505 from benoitc/feature/gunicornc-control-interface
feat(ctl): add gunicornc control interface
2026-02-13 11:31:45 +01:00
Benoit Chesneau
63df19bd5c fix(tests): use process groups for reliable signal handling in PyPy
- Use preexec_fn=os.setsid to create new process group
- Send signals to process group with os.killpg() instead of single process
- Add explicit timeout and graceful-timeout to gunicorn command
- Fixes test failures on PyPy 3.10 where signals weren't propagating properly
2026-02-13 11:02:10 +01:00
Benoit Chesneau
cd77bcc941 fix(tests): increase wait time for all server tests 2026-02-13 10:48:41 +01:00
Benoit Chesneau
02ea9855c1 fix(tests): improve server test reliability on FreeBSD 2026-02-13 10:37:22 +01:00
Benoit Chesneau
6d81c9ebcd fix: resolve pylint warnings 2026-02-13 02:40:01 +01:00
Benoit Chesneau
7486baa0ad fix: remove unused imports 2026-02-13 02:35:02 +01:00
Eng Zer Jun
54eb411d34
Enable hard_linebreaks
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2026-02-13 09:31:17 +08:00
Benoit Chesneau
3e60d2942d docs: add gunicornc control interface guide
- Add guides/gunicornc.md with usage examples and command reference
- Update mkdocs.yml navigation to include Control Interface guide
- Update 2026-news.md and news.md changelog with 25.2.0 release
- Regenerate reference/settings.md with control socket settings
2026-02-13 02:29:44 +01:00