3218 Commits

Author SHA1 Message Date
Tanmoy Sarkar
ef6e42ecc1 feat(companion): Implement status, start, stop, and restart commands 2026-06-09 21:44:27 +05:30
Tanmoy Sarkar
104bfcebdd feat(companion): Add Unix control socket and JSON command protocol
Add gunicorn/companion/control.py with ControlServer, the manager's control
endpoint. It owns the Unix socket lifecycle (create unlinks any stale socket,
binds, chmods 0o600, and listens; close cleans up) and the newline-delimited
JSON framing: serve_connection buffers reads and answers each complete line.
decode_command parses a request into a JSON object carrying a string cmd, and
encode_response writes a newline-terminated JSON line; malformed input becomes
a CommandError rendered as an {ok: false, error: ...} reply so a bad client
can't take the manager down. Turning a command into an action is delegated to a
dispatch callable, wired up in the later command tasks.

The socket is 0o600 and owned by the non-root user gunicorn runs as; no group
switching.

Add tests/test_companion_control.py covering decode, encode, handle_line
dispatch and error envelopes, and socket create/close.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 18:23:03 +05:30
Tanmoy Sarkar
c82df2ab94 feat(companion): Make manual_stop ownership explicit
spawn_process no longer clears manual_stop; spawning is now policy-neutral.
Clearing the flag is owned by start_process and restart_process (which already
do it), and the respawn paths (retry_backoff, restart_pending) only run when
the flag is already false. A manually stopped companion now keeps manual_stop
set through its exit, so it settles in STOPPED and is not auto-restarted.

Add tests: manual_stop preserved through exit, start clears it, spawn leaves
it untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 18:17:44 +05:30
Tanmoy Sarkar
8e0ca34277 feat(companion): Implement restart_process control command
Add restart_process(name) following supervisor's restart rules: it always
clears manual_stop. RUNNING/STARTING are sent their stop_signal and enter
STOPPING with restart_pending set and a deadline from reload_timeout; the
reaper respawns them immediately once the old child exits. BACKOFF and STOPPED
start again right away. STOPPING is rejected. It never rereads config.

handle_exit now honors restart_pending first, respawning immediately (bumping
restart_count) instead of going to STOPPED or BACKOFF. Add a restart_pending
field on CompanionProcess.

Add tests for the running, pending-reap, stopped, backoff, and stopping cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 18:10:40 +05:30
Tanmoy Sarkar
8d9eb76e3d feat(companion): Implement stop_process control command
Add stop_process(name) following supervisor's stop rules: it always sets
manual_stop so the companion will not auto-restart. RUNNING/STARTING are sent
their stop_signal and moved to STOPPING with a stop_deadline (now +
stop_timeout) for the run loop to reap or SIGKILL; BACKOFF cancels its pending
retry and settles in STOPPED; STOPPED and STOPPING are success no-ops. Add
_signal_number to resolve a signal name and a stop_deadline field on
CompanionProcess.

Add tests for the running, backoff, already-stopped, unknown, and signal-name
cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 18:06:58 +05:30
Tanmoy Sarkar
8c9aa962ae feat(companion): Implement start_process control command
Add start_process(name) following supervisor's start rules: STOPPED and
BACKOFF clear manual_stop, drop any pending retry, and spawn now; RUNNING and
STARTING report success without acting; STOPPING is rejected so the caller
retries. Returns (ok, message).

Add tests for the stopped, backoff, running, stopping, and unknown cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 17:59:35 +05:30
Tanmoy Sarkar
87bc4cf70e feat(companion): Implement BACKOFF with fixed restart delay
Reaping now transitions each exited companion via handle_exit: a manually
stopped one settles in STOPPED, any other exit enters BACKOFF with
next_retry_at = now + restart_delay (fixed, no exponential backoff or cap).
Add retry_backoff to re-fork BACKOFF companions once their delay elapses,
bumping restart_count and returning them to STARTING.

Add tests for backoff on unexpected exit, manual-stop staying stopped, retry
timing, and reap-to-backoff.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 17:56:22 +05:30
Tanmoy Sarkar
84d69c46fd feat(companion): Promote companions from STARTING to RUNNING after startsecs
Add promote_running to CompanionManager: scans STARTING companions and moves
any that have stayed alive at least their startsecs window to RUNNING, logging
the pid and returning the promoted ones. Companions that die inside the window
are left to reaping.

Add tests for promotion after the window, too-early no-op, and non-STARTING.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 17:52:25 +05:30
Tanmoy Sarkar
bd8a91f656 feat(companion): Reap exited companion processes
Add reap_processes to CompanionManager: drains waitpid(WNOHANG), matches each
dead pid back to its companion, and records the exit via _record_exit (signal
number or exit code, exited_at, exit_count) while freeing the pid. Returns the
reaped companions; the restart decision stays with the run loop.

Add tests for exit-code, signal, and no-children cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 17:49:27 +05:30
Tanmoy Sarkar
2bf7e1b1fb feat(companion): Redirect companion stdout and stderr
Child calls _redirect_output after env setup: each configured log path is
opened append-mode and dup2'd onto fd 1/2. None/inherit keeps the inherited
fd; stderr stdout shares stdout's fd. Rotation stays external.

Add tests for inherit, append flags, file dup2, and stderr-to-stdout.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 17:30:18 +05:30
Tanmoy Sarkar
ea2748a209 feat(companion): Apply cwd and env in spawned companion child
Child runs _apply_environment before the target: os.chdir(cwd) then
os.environ.update(env).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 17:06:42 +05:30
Tanmoy Sarkar
5639d467f3 feat(companion): Add CompanionManager skeleton and single-companion spawn
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 16:57:18 +05:30
Tanmoy Sarkar
78d67197b6 feat(companion): Add CompanionProcess runtime state and status helpers 2026-06-09 16:34:02 +05:30
Tanmoy Sarkar
2241dd4031 feat(companion): Add states and CompanionConfig with config hash 2026-06-09 15:35:00 +05:30
Tanmoy Sarkar
3f479157d7 feat(companion): Add companion process config settings 2026-06-09 15:17:43 +05:30
Tanmoy Sarkar
6fd0c2b236 feat: Add plan for companion process manager 2026-06-09 14:25:28 +05:30
Ankush Menat
54b59ca884
fix: Avoid queue time in slow req prediction (#13)
* fix: Avoid queue time in slow req prediction

Else we punish requests that were blocked by slow requests in queue.

* fix: add basic logging for adaptive queueing
2026-05-29 10:47:16 +05:30
Ankush Menat
f61e5845f3
Merge pull request #12 from frappe/multi_queue
feat: Adaptive queueing of requests for gthread
2026-05-28 17:07:53 +05:30
Ankush Menat
c20f3a3784 fix: Support configuring via envvar 2026-05-28 16:59:47 +05:30
Ankush Menat
c24711b795 fix: Increase default slow request threshold 2026-05-28 16:59:45 +05:30
Ankush Menat
7dc4d184be refactor: Simpler implementation
Just do multiple queues, nothing else.
2026-05-28 16:33:54 +05:30
Ankush Menat
ec6af68013 fix: Remove hardcoded paths for slow prediction 2026-05-28 16:01:02 +05:30
Ankush Menat
2471050b3a refactor: Use only 2 pool variables. 2026-05-28 15:54:33 +05:30
Ankush Menat
48260712ea fix: Limit peek to 8KB 2026-05-28 15:53:39 +05:30
Ankush Menat
ee9bf1e950 feat: Adaptive queueing of slow/fast requests 2026-05-27 11:58:54 +05:30
Ankush Menat
bb554053bb
Merge branch 'benoitc:master' into master 2025-05-07 22:04:50 +05:30
Ankush Menat
6f9b8a5a7e fix: use OSError instead of deprecated EnvironmentError 2025-05-07 22:03:04 +05:30
Ankush Menat
6b60bcff7e
Revert "gthread: only read sockets when they are readable" (#7)
* Revert "gthread: only read sockets when they are readable"

This reverts commit 0ebb73aa240f0ecffe3e0922d54cfece19f5bfed.

* chore: update readme
2025-05-07 19:38:52 +05:30
Ankush Menat
ca67a87c37
chore: update readme 2025-05-07 19:38:21 +05:30
Ankush Menat
73eae94e8b Revert "gthread: only read sockets when they are readable"
This reverts commit 0ebb73aa240f0ecffe3e0922d54cfece19f5bfed.
2025-05-07 19:28:08 +05:30
Benoit Chesneau
a86ea1e4e6
Merge pull request #3355 from mondwan/patch-1
Fix typo in the settings.rst
2025-03-20 21:19:12 +01:00
Mond WAN
d05b6c5425 Fix typo in the getgrnam
getgrnam is under grp module instead of pwd
2025-03-14 18:01:52 +00:00
Mond WAN
1bc351e794 Revert "Fix typo in the settings.rst"
This reverts commit 02e5f64c76db6d33ffb22448a7f35d7aececf51d.
2025-03-14 18:00:02 +00:00
Mond WAN
02e5f64c76
Fix typo in the settings.rst
getgrnam is under grp module instead of pwd
2025-03-11 11:16:17 +00:00
Ankush Menat
72c1e495d8
fix(DX): Print traceback of timed out requests (#6) 2025-02-07 19:51:15 +05:30
Ankush Menat
b9a8570039
fix: Drain shutdown event (#5)
Otherwise it goes in a spinny loop
2025-02-03 11:15:01 +05:30
Ankush Menat
4f74f4a4b8 fix: fast shutdown (#4)
Break the poller wait by using shutdown event to let it know its time to wake up.
2025-01-29 11:59:39 +05:30
Ankush Menat
2f5b492339
docs: update readme 2025-01-13 19:07:33 +05:30
Ankush Menat
e5d5bc4e35
perf: increase selector timeout (#2) 2025-01-13 19:00:53 +05:30
Ankush Menat
0c9b266790
fix: Request timeout for gthread workers (#1)
* ci: drop unsupported versions

* fix: a *dirty* request timeout for gthread workers

* fix: drain current futures before exiting

* fix: avoid accepting new connections completely

* refactor: Just rely on graceful_timeout

This has minor side effect of no progress on accepted BUT not "read"
reqeusts. But I feel this makes code simpler and less prone to bugs.

These time outs should be *rare* and fixed anyway.
2025-01-13 18:44:27 +05:30
Benoit Chesneau
bacbf8aa51
Merge pull request #2938 from Affirm/reuse-port-fix
Fix reuse-port to balance requests across Gunicorn workers
2024-10-25 11:36:15 +02:00
Benoit Chesneau
903792f152
Merge pull request #3269 from pajod/patch-include-tox.ini-in-sdist
packaging: include tox.ini in sdist
2024-08-13 21:52:07 +02:00
Paul J. Dorn
0c722b6915 packaging: add .pylintrc to sdist
undo when .pylintrc is moved into pyproject.toml (which is only
supported since pylint 2.5.3+ @ 2020-06-8)
2024-08-12 16:00:56 +02:00
Paul J. Dorn
81c1fc952a packaging: include tox.ini in sdist
tox testing works without full git repo, so no obvious harm in
aiding people running it from source distribution

based on setuptools itself doing it, and suggesting so in an example in
docs:
https://setuptools.pypa.io/en/latest/userguide/miscellaneous.html#controlling-files-in-the-distribution
551eb7f444

beware of .gitattributes / MANIFEST.in precedence when using setuptools-scm
2024-08-12 15:45:52 +02:00
benoitc
7268a61099 fix util.set_owner_process
Python 2.x is not supported anymore, let's remove this extra feature.

fix #3212
2024-08-11 09:52:51 +02:00
Benoit Chesneau
5aeb0652e1
Merge pull request #2887 from python273/patch-1
Fix sendfile.py example
2024-08-11 00:31:24 +02:00
benoitc
411986d619 fix doc 2024-08-10 22:34:28 +02:00
Benoit Chesneau
334392e779
Merge pull request #2559 from laggardkernel/bugfix/reexec-env
Typo fix for reexec related environ variables
2024-08-10 11:07:03 +02:00
Benoit Chesneau
e75c3533e3
Merge pull request #3189 from pajod/patch-py36
chore: eat Python 2 leftovers
2024-08-10 10:40:40 +02:00
benoitc
9357b28dd8 keep document user in access_log_format setting 2024-08-10 10:19:36 +02:00