218 Commits

Author SHA1 Message Date
Benoit Chesneau
af8897a14c Add pytest and requests to embedding_service Dockerfile
Include test dependencies in Docker image for testing.
2026-03-23 13:37:02 +01:00
Benoit Chesneau
709a6ad159
feat(dirty): add stash - global shared state between workers (#3503)
* feat(dirty): add stash - global shared state between workers

Add a simple key-value store (stash) that allows dirty workers to share
state through the arbiter. Tables are stored directly in arbiter memory
for fast access and simplicity.

Features:
- Auto-create tables on first access
- Dict-like interface via stash.table()
- Pattern matching for keys (glob patterns)
- Module-level API: stash.put(), stash.get(), stash.delete(), etc.

Usage:
    from gunicorn.dirty import stash

    stash.put("sessions", "user:1", {"name": "Alice"})
    user = stash.get("sessions", "user:1")

    # Or dict-like
    sessions = stash.table("sessions")
    sessions["user:1"] = {"name": "Alice"}

New files:
- gunicorn/dirty/stash.py - Client API and StashTable class
- Protocol additions for MSG_TYPE_STASH and STASH_OP_* codes

Note: Tables are ephemeral - lost if arbiter restarts.

* test(dirty): add tests for stash protocol and encoding

Test coverage for:
- Stash message creation and encoding
- Protocol constants (MSG_TYPE_STASH, STASH_OP_*)
- Error classes (StashError, StashTableNotFoundError, StashKeyNotFoundError)
- StashTable dict-like interface
- Edge cases: unicode, complex values, special patterns

* example(dirty): add stash usage example and integration tests

- Add SessionApp to dirty_app.py demonstrating stash usage
- Add /session/* endpoints to wsgi_app.py
- Add test_stash_integration.py with Docker tests
- Update docker-compose.yml with stash-test service
- Fix: Set GUNICORN_DIRTY_SOCKET in dirty arbiter for worker access

* docs(dirty): add stash documentation
2026-02-12 21:45:49 +01:00
Benoit Chesneau
415aa77343 fix: revert embedding_service port to 8000 for CI 2026-02-11 23:54:36 +01:00
Benoit Chesneau
6d691b30e1 chore: use different ports in example docker-compose files
Avoid port conflicts when running multiple examples:
- dirty_example: 8001
- embedding_service: 8002
- celery_alternative: 8003
2026-02-11 23:40:10 +01:00
Benoit Chesneau
c0cc8c0de0 test(dirty): add Docker setup for dirty example integration tests
- Add Dockerfile and docker-compose.yml for running examples in containers
- Add test_integration.py for HTTP-level integration testing
- Update test_worker_integration.py to use MockWriter for handle_request
- Use integer request IDs for binary protocol compatibility
- Add GUNICORN_BIND env var support in gunicorn_conf.py for Docker
2026-02-11 23:30:48 +01:00
Benoit Chesneau
00da70292f docs(dirty): update examples and docs for binary protocol
- Update test_protocol.py example to use binary protocol
- Add test_binary_data_handling example showing raw bytes transfer
- Update dirty.md to document binary TLV protocol format
- Replace JSON references with binary protocol
- Add Binary Protocol section with header and TLV encoding details
2026-02-11 23:15:14 +01:00
Benoit Chesneau
95b7ffeeaa chore: prepare release 25.0.2
- Bump version to 25.0.2
- Update copyright year to 2026 in LICENSE and NOTICE
- Add license headers to all Python source files
- Add changelog entry for 25.0.2
2026-02-06 08:21:18 +01:00
Benoit Chesneau
ce352dc230 fix(asgi): add chunked transfer encoding for streaming responses
Add chunked transfer encoding support to the ASGI worker for HTTP/1.1
streaming responses that don't have a Content-Length header. This fixes
SSE (Server-Sent Events) connections not closing properly.

Without chunked encoding or Content-Length, HTTP/1.1 clients wait for
the connection to close to determine end-of-response, causing streaming
endpoints to hang.

Also updates the celery_alternative example to use FastAPI with the
native ASGI worker and uvloop, demonstrating async task execution with
proper SSE streaming.
2026-02-02 10:13:33 +01:00
Benoit Chesneau
be54850ff8 docs: rename example title to Celery Alternative 2026-02-01 16:54:22 +01:00
Benoit Chesneau
17ac6a5254 examples: add celery_alternative example using dirty arbiters
Demonstrates replacing Celery with Gunicorn's dirty arbiters for
background task processing. Includes:

- 4 task workers: Email, Image, Data, Scheduled
- Stateful workers with persistent connections/caches
- Streaming progress for long-running tasks
- Per-app worker allocation
- Flask API with 15+ endpoints
- Docker deployment (single container vs Celery's 4+)
- Unit tests (19 tests) and integration tests
- Migration guide from Celery
2026-02-01 16:51:48 +01:00
Benoit Chesneau
315e7bde80 fix(http2): ALPN negotiation for gevent/eventlet workers
- Add explicit do_handshake() in base_async.py before ALPN check
  when do_handshake_on_connect is False
- Mark eventlet worker as deprecated (removal in 26.0)
- Add HTTP/2 gevent example with Docker and tests
- Update documentation to reflect eventlet deprecation
- Remove eventlet websocket example (gevent version exists)

The ALPN fix ensures HTTP/2 works correctly with gevent and eventlet
workers when do_handshake_on_connect config is False (the default).
Without explicit handshake, selected_alpn_protocol() returns None.
2026-01-28 13:42:48 +01:00
Benoit Chesneau
fa5e319f15 docs(http2): add ASGI example demonstrating priority and trailers
Add complete HTTP/2 example in examples/http2_features/:
- ASGI app showing priority access and trailer sending
- Test script using raw h2 library for HTTP/2 testing
- Docker setup for easy testing
- Documentation update referencing the example

The example demonstrates:
- Reading http.response.priority extension in ASGI scope
- Sending http.response.trailers messages
- Multiple streams on the same connection
2026-01-27 13:15:36 +01:00
Benoit Chesneau
cc39ed922e examples(dirty): add streaming chat demo with SSE
Add a lightweight chat simulator demonstrating dirty worker streaming:
- Token-by-token SSE streaming via async generators
- FastAPI endpoint with browser UI
- Multiple canned responses based on keywords
- Docker deployment with docker-compose
- Integration tests for SSE protocol

Update docs/content/dirty.md to link to both examples.
2026-01-25 10:26:12 +01:00
Benoit Chesneau
0e05c824e9 feat(examples): add FastAPI embedding service with Docker testing
Add a complete example demonstrating dirty workers with sentence-transformers
for text embeddings via FastAPI:

- EmbeddingApp DirtyApp that loads and manages the ML model
- FastAPI endpoints for /embed and /health
- Docker and docker-compose configuration
- Integration tests with numpy similarity checks
- GitHub Actions CI workflow
2026-01-25 10:23:25 +01:00
Benoit Chesneau
77222b8017 feat: add dirty arbiters for long-running blocking operations
Introduce Dirty Arbiters - a separate process pool for executing
long-running, blocking operations (AI model loading, heavy computation)
without blocking HTTP workers. Inspired by Erlang's dirty schedulers.

Key features:
- Completely separate from HTTP workers - can be killed/restarted independently
- Stateful - loaded resources persist in dirty worker memory
- Message-passing IPC via Unix sockets with JSON serialization
- Explicit execute() API from HTTP workers
- Asyncio-based for clean concurrent handling

Architecture:
- DirtyArbiter: manages the dirty worker pool, routes requests
- DirtyWorker: executes functions, maintains state, handles requests
- DirtyClient: sync/async API for HTTP workers to call dirty apps
- DirtyProtocol: length-prefixed JSON messages over Unix sockets
- DirtyApp: base class for dirty applications

Configuration options:
- dirty_apps: list of import paths for dirty applications
- dirty_workers: number of dirty workers (default: 0)
- dirty_timeout: task timeout in seconds (default: 300)
- dirty_graceful_timeout: shutdown timeout (default: 30)

Lifecycle hooks:
- on_dirty_starting(arbiter)
- dirty_post_fork(arbiter, worker)
- dirty_worker_init(worker)
- dirty_worker_exit(arbiter, worker)

Includes comprehensive test suite with 164 tests covering:
- Protocol encoding/decoding
- Worker and arbiter lifecycle
- Client sync/async APIs
- Signal handling
- Error handling and timeouts
- Integration tests
2026-01-25 10:21:18 +01:00
Benoit Chesneau
7894d1c170 release: prepare 24.1.0
- Bump version to 24.1.0
- Add PROXY protocol v2 documentation to deploy guide
- Add 24.1.0 changelog with new features and bug fixes
- Update all docs.gunicorn.org URLs to gunicorn.org
2026-01-23 18:47:17 +01:00
Benoit Chesneau
4b9d787c93 tornado: Require Tornado 6.5.0+ for security fixes
Update minimum Tornado version to 6.5.0 to address:
- CVE-2024-52804 (Medium): HTTP Cookie Parsing DoS
- CVE-2025-47287 (High 7.5): Multipart/Form-Data Parser DoS

This simplifies the tornado worker by removing legacy code paths
for Tornado < 5.0 and < 6.0, reducing the codebase by ~30%.

Changes:
- pyproject.toml: Update tornado requirement to >=6.5.0
- gtornado.py: Remove TORNADO5 constant and legacy code paths
- tornadoapp.py: Update example to use async/await syntax
- test_gtornado.py: Add comprehensive test suite
2026-01-23 00:02:01 +01:00
Benoit Chesneau
ae1eea8108 asgi: Add native ASGI worker with HTTP and WebSocket support
Add a new ASGI worker type that provides native async support using
gunicorn's own HTTP parsing infrastructure adapted for asyncio.

Features:
- HTTP/1.1 with keepalive support
- WebSocket connections (RFC 6455)
- ASGI lifespan protocol for startup/shutdown hooks
- Optional uvloop support for improved performance
- Full proxy protocol support (inherited from gunicorn)

New configuration options:
- --asgi-loop: Event loop selection (auto/asyncio/uvloop)
- --asgi-lifespan: Lifespan protocol control (auto/on/off)
- --root-path: ASGI root path for reverse proxy setups

Usage: gunicorn -k asgi myapp:app
2026-01-22 17:05:29 +01:00
Benoit Chesneau
5aeb0652e1
Merge pull request #2887 from python273/patch-1
Fix sendfile.py example
2024-08-11 00:31:24 +02:00
Benoit Chesneau
e75c3533e3
Merge pull request #3189 from pajod/patch-py36
chore: eat Python 2 leftovers
2024-08-10 10:40:40 +02:00
Benoit Chesneau
5f820ce37c
Merge pull request #3167 from tbascoul/tiny-example-config-fix
update example_config with latest settings
2024-08-07 00:19:34 +02:00
Benoit Chesneau
5e39f88622
Update tornadoapp.py 2024-08-06 17:50:16 +02:00
Paul J. Dorn
4f7766585d Python 3.3+: replace deprecated aliases
since 3.3: EnvironmentError, IOError, socket.error and select.error are merged into IOError.
They may now return a more specific subclass - which this commit does not utilize yet.
2024-04-22 03:33:31 +02:00
Paul J. Dorn
422b18acea class Name(object): -> class Name: 2024-04-22 03:33:30 +02:00
Paul J. Dorn
4323027b1e drop long-default - coding: utf-8 2024-04-22 03:33:14 +02:00
Theo Bascoul
1954fb63f1 update example_config with latest settings 2024-03-18 16:14:14 +01:00
Kirill
ea57dfec98
Fix sendfile.py example 2022-10-27 19:21:10 +03:00
Tero Saarni
362a52bd84 Added parameter to ssl_context hook for constructing default context
Signed-off-by: Tero Saarni <tero.saarni@est.tech>
2022-02-11 19:26:56 +02:00
Tero Saarni
5a581c0b14 Update SSLContext handling
* Change deprecated ssl.wrap_socket() to SSLContext.wrap_context().
* Add new server hook to allow user to create custom SSLContext.
* Updated the documentation.

Signed-off-by: Tero Saarni <tero.saarni@est.tech>
2021-09-12 17:39:06 +03:00
Hugo van Kemenade
45687c358c Replace deprecated unittest alias 2021-08-18 13:09:03 +03:00
Christian Clauss
7e16d13011
Port websocket examples to Python 3 (#2371) 2020-07-02 23:34:11 +03:00
Tim Gates
f626830cde
Fix simple typo: convertable -> convertible
There is a small typo in examples/websocket/gevent_websocket.py, examples/websocket/websocket.py.
Should read `convertible` rather than `convertable`.
2020-03-01 19:14:18 +11:00
Brett Randall
c1bea68fce
Added a set of pip requirements files to cover the examples in examples/frameworks.
Signed-off-by: Brett Randall <javabrett@gmail.com>
2020-01-14 06:25:46 +11:00
benoitc
499dcd4021 add a simple example to test module path 2019-11-21 10:18:07 +01:00
Benoit Chesneau
e147feaf8b fix echo example on python 3.7 2019-09-27 23:15:59 +02:00
Brett Randall
879651bb6f Header values are encoded using latin-1, not ascii.
This commit reverts one aspect changed by 5f4ebd2eb2b08783a5fbefe79d09fcb3fc1fbc73 (#1151);
header-values are again encoded as latin-1 and not ascii. Test is restored but uses
a latin-1-mappable test-character, not a general utf8 character.

Fixed #1778.

Signed-off-by: Brett Randall <javabrett@gmail.com>
2019-04-18 04:23:19 +03:00
Marat
a8963ef1a5 Remove redundant super() arguments 2019-03-16 15:00:07 -07:00
Benoit Chesneau
fe7632fe37 make sure all examples work on python 3
fix #1961
2019-01-24 23:19:38 +01:00
Benoit Chesneau
c16abf922e fix example to work on python3
* fix longpoll example

* fix websocket example

* fix django example

* fix flask exampl
2018-09-04 12:18:05 +02:00
Hugo
e974f30517 Drop support for Python 2
Co-Authored-By: Dustin Ingram <di@users.noreply.github.com>
Co-Authored-By: Berker Peksag <berker.peksag@gmail.com>
2018-08-01 15:31:17 +03:00
Randall Leeds
7891310d80 Use consistent style in nginx.conf 2018-06-17 13:19:42 -07:00
Patrick
66ec021291
All good
… final edits.
2018-05-19 00:54:48 -07:00
Patrick
a80052e66a
Changes based on feedback
I also changed the location for access.log
2018-05-19 00:01:05 -07:00
Patrick
a047d4f23f
Pid and Logging changes
PID didn't work at all on Ubuntu 16.04 with the provided sample, my commented out change does, also added logging change to conform to what is out of the box.
2018-05-16 12:11:18 -07:00
Berker Peksag
01a1c7ca9a
Add an example for the raw_env setting
Refs #1472
2018-03-27 14:29:30 +03:00
Randall Leeds
b07532be75 Forbid contradictory secure scheme headers
When a request specifies contradictory secure scheme headers, raise a
parse error.
2018-01-10 12:10:35 -08:00
Berker Peksag
2dd7321c90
Deprecate gaiohttp worker and document alternatives (#1569)
Fixes #1338
2017-10-31 07:30:41 +03:00
Berker Peksag
31b8e48a78 Silence prospector warnings 2017-02-07 09:05:43 +03:00
Berker Peksag
9fad8ab6f5 Remove outdated documentation about using entry points
Closes #1365
2017-02-05 01:24:49 +03:00
Berker Peksag
6dbbaa595b Modernize CherryPy example 2016-09-17 13:23:30 +03:00