jingrow/gunicorn
1
0
Fork 0
mirror of https://github.com/frappe/gunicorn.git synced 2026-01-14 11:09:11 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'develop'

Browse Source
This commit is contained in:
benoitc 2012-11-19 11:49:05 +01:00
commit 0b40e69e67
79 changed files with 1072 additions and 855 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
.travis.yml
View File

@ -3,10 +3,15 @@ language: python
python: python:
- 2.6 - 2.6
- 2.7 - 2.7
- 3.2
- 3.3
- pypy - pypy
install: python setup.py install install:
script: nosetests - pip install -r requirements_dev.txt --use-mirrors
- python setup.py install
script: py.test tests/
branches: branches:
only: only:

8
Makefile
View File

@ -1,15 +1,13 @@
build: build:
virtualenv --no-site-packages . virtualenv --no-site-packages .
bin/python setup.py develop bin/python setup.py develop
bin/pip install coverage bin/pip install -r requirements_dev.txt
bin/pip install nose
test: test:
bin/nosetests bin/python setup.py test
coverage: coverage:
bin/nosetests --with-coverage --cover-html --cover-html-dir=html \ bin/python setup.py test --cov
--cover-package=gunicorn
clean: clean:
@rm -rf .Python bin lib include man build html @rm -rf .Python bin lib include man build html

22
README.rst
View File

@ -153,6 +153,28 @@ And then as per usual::
$ cd yourpasteproject $ cd yourpasteproject
$ paster serve development.ini workers=2 $ paster serve development.ini workers=2
**Gunicorn paster from script**
If you'd like to run Gunicorn paster from a script instead of the command line (for example: a runapp.py to start a Pyramid app),
you can use this example to help get you started::
import os
import multiprocessing
from paste.deploy import appconfig, loadapp
from gunicorn.app.pasterapp import paste_server
if __name__ == "__main__":
iniFile = 'config:development.ini'
port = int(os.environ.get("PORT", 5000))
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent'
app = loadapp(iniFile, relative_to='.')
paste_server(app, host='0.0.0.0', port=port, workers=workers, worker_class=worker_class)
LICENSE LICENSE
------- -------

4
THANKS
View File

@ -45,3 +45,7 @@ Caleb Brown <git@calebbrown.id.au>
Marc Abramowitz <marc@marc-abramowitz.com> Marc Abramowitz <marc@marc-abramowitz.com>
Vangelis Koukis <vkoukis@grnet.gr> Vangelis Koukis <vkoukis@grnet.gr>
Prateek Singh Paudel <pratykschingh@gmail.com> Prateek Singh Paudel <pratykschingh@gmail.com>
Andrew Gorcester <andrewsg@terra.(none)>
Kenneth Reitz <me@kennethreitz.com>
Eric Shull <eric@elevenbasetwo.com>
Christos Stavrakakis <cstavr@grnet.gr>

17
docs/site/css/style.css
View File

@ -60,6 +60,15 @@ a:hover {
border-bottom: 1px solid #2A8729; border-bottom: 1px solid #2A8729;
} }
.latest {
width: 150px;
top: 0;
display: block;
float: right;
font-weight: bold;
}
.logo-div { .logo-div {
width: 1000px; width: 1000px;
margin: 0 auto; margin: 0 auto;
@ -283,12 +292,14 @@ a:hover {
margin: 0 0 9px; margin: 0 0 9px;
} }
.tab-box a { .tab-box a,
.latest a {
color: #3F3F27; color: #3F3F27;
text-decoration: underline; text-decoration: underline;
} }
.tab-box a:hover { .tab-box a:hover,
.latest a:hover {
color: #1D692D; color: #1D692D;
} }
@ -389,4 +400,4 @@ pre {
.footer-wp a:hover { .footer-wp a:hover {
color: #1D692D; color: #1D692D;
} }

9
docs/site/index.html
View File

@ -6,10 +6,19 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<link rel="stylesheet" type="text/css" href="css/style.css" /> <link rel="stylesheet" type="text/css" href="css/style.css" />
<link rel="shortcut icon" href="images/favicon.png" type="image/x-icon"> <link rel="shortcut icon" href="images/favicon.png" type="image/x-icon">
<link rel="alternate"
href="https://github.com/benoitc/gunicorn/commits/master.atom"
type="application/atom+xml" title="Gunicorn commits">
</head> </head>
<body> <body>
<div class="logo-wrapper"> <div class="logo-wrapper">
<div class="logo-div"> <div class="logo-div">
<div class="latest">
Latest version: <strong><a
href="http://docs.gunicorn.org/en/latest/news.html#id1">0.15.0</a></strong>
</div>
<div class="logo"><img src="images/logo.jpg" ></div> <div class="logo"><img src="images/logo.jpg" ></div>
</div> </div>
</div> </div>

11
docs/source/configure.rst
View File

@ -20,6 +20,17 @@ Once again, in order of least to most authoritative:
2. Configuration File 2. Configuration File
3. Command Line 3. Command Line
.. note::
To check your configuration when using the command line or the
configuration file you can run the following command::
$ gunicorn --check-config
It will also allows you to know if your applican can be launched.
Framework Settings Framework Settings
================== ==================

12
docs/source/news.rst
View File

@ -1,6 +1,18 @@
Changelog Changelog
========= =========
0.16.0 / develop
----------------
- **Added support for Python 3.2 & 3.3**
- Expose --pythonpath command to all gunicorn commands
- Honor $PORT environment variable, useful for deployement on heroku
- Removed support for Python 2.5
- Make sure we reopen teh logs on the console
- Fix django settings module detection from path
- Reverted timeout for client socket. Fix issue on blocking issues.
- Fixed gevent worker
0.15.0 / 2012-10-18 0.15.0 / 2012-10-18
------------------- -------------------

3
docs/source/settings.rst
View File

@ -482,6 +482,9 @@ The Python path to a Django settings module.
e.g. 'myproject.settings.main'. If this isn't provided, the e.g. 'myproject.settings.main'. If this isn't provided, the
DJANGO_SETTINGS_MODULE environment variable will be used. DJANGO_SETTINGS_MODULE environment variable will be used.
Server Mechanics
----------------
pythonpath pythonpath
~~~~~~~~~~ ~~~~~~~~~~

24
examples/frameworks/django/djangotest/testing/views.py
View File

@ -12,18 +12,18 @@ class MsgForm(forms.Form):
subject = forms.CharField(max_length=100) subject = forms.CharField(max_length=100)
message = forms.CharField() message = forms.CharField()
f = forms.FileField() f = forms.FileField()
def home(request): def home(request):
from django.conf import settings from django.conf import settings
print settings.SOME_VALUE print(settings.SOME_VALUE)
subject = None subject = None
message = None message = None
size = 0 size = 0
print request.META print(request.META)
if request.POST: if request.POST:
form = MsgForm(request.POST, request.FILES) form = MsgForm(request.POST, request.FILES)
print request.FILES print(request.FILES)
if form.is_valid(): if form.is_valid():
subject = form.cleaned_data['subject'] subject = form.cleaned_data['subject']
message = form.cleaned_data['message'] message = form.cleaned_data['message']
@ -31,29 +31,29 @@ def home(request):
size = int(os.fstat(f.fileno())[6]) size = int(os.fstat(f.fileno())[6])
else: else:
form = MsgForm() form = MsgForm()
return render_to_response('home.html', { return render_to_response('home.html', {
'form': form, 'form': form,
'subject': subject, 'subject': subject,
'message': message, 'message': message,
'size': size 'size': size
}, RequestContext(request)) }, RequestContext(request))
def acsv(request): def acsv(request):
rows = [ rows = [
{'a': 1, 'b': 2}, {'a': 1, 'b': 2},
{'a': 3, 'b': 3} {'a': 3, 'b': 3}
] ]
response = HttpResponse(mimetype='text/csv') response = HttpResponse(mimetype='text/csv')
response['Content-Disposition'] = 'attachment; filename=report.csv' response['Content-Disposition'] = 'attachment; filename=report.csv'
writer = csv.writer(response) writer = csv.writer(response)
writer.writerow(['a', 'b']) writer.writerow(['a', 'b'])
for r in rows: for r in rows:
writer.writerow([r['a'], r['b']]) writer.writerow([r['a'], r['b']])
return response return response

2
examples/frameworks/django/testing/testing/apps/someapp/middleware.py
View File

@ -4,7 +4,7 @@ import gevent
def child_process(queue): def child_process(queue):
while True: while True:
print queue.get() print(queue.get())
requests.get('http://requestb.in/15s95oz1') requests.get('http://requestb.in/15s95oz1')
class GunicornSubProcessTestMiddleware(object): class GunicornSubProcessTestMiddleware(object):

14
examples/frameworks/django/testing/testing/apps/someapp/views.py
View File

@ -1,6 +1,7 @@
# Create your views here. # Create your views here.
import csv import csv
import io
import os import os
from django import forms from django import forms
from django.http import HttpResponse from django.http import HttpResponse
@ -16,22 +17,27 @@ class MsgForm(forms.Form):
def home(request): def home(request):
from django.conf import settings from django.conf import settings
print settings.SOME_VALUE print(settings.SOME_VALUE)
subject = None subject = None
message = None message = None
size = 0 size = 0
print request.META print(request.META)
if request.POST: if request.POST:
form = MsgForm(request.POST, request.FILES) form = MsgForm(request.POST, request.FILES)
print request.FILES print(request.FILES)
if form.is_valid(): if form.is_valid():
subject = form.cleaned_data['subject'] subject = form.cleaned_data['subject']
message = form.cleaned_data['message'] message = form.cleaned_data['message']
f = request.FILES['f'] f = request.FILES['f']
if not hasattr(f, "fileno"): if not hasattr(f, "fileno"):
size = len(f.read()) size = len(f.read())
else: else:
size = int(os.fstat(f.fileno())[6]) try:
size = int(os.fstat(f.fileno())[6])
except io.UnsupportedOperation:
size = len(f.read())
else: else:
form = MsgForm() form = MsgForm()

10
examples/longpoll.py
View File

@ -1,13 +1,14 @@
# -*- coding: utf-8 - # -*- coding: utf-8 -
# #
# This file is part of gunicorn released under the MIT license. # This file is part of gunicorn released under the MIT license.
# See the NOTICE for more information. # See the NOTICE for more information.
import sys
import time import time
class TestIter(object): class TestIter(object):
def __iter__(self): def __iter__(self):
lines = ['line 1\n', 'line 2\n'] lines = ['line 1\n', 'line 2\n']
for line in lines: for line in lines:
@ -16,12 +17,13 @@ class TestIter(object):
def app(environ, start_response): def app(environ, start_response):
"""Application which cooperatively pauses 20 seconds (needed to surpass normal timeouts) before responding""" """Application which cooperatively pauses 20 seconds (needed to surpass normal timeouts) before responding"""
data = 'Hello, World!\n' data = b'Hello, World!\n'
status = '200 OK' status = '200 OK'
response_headers = [ response_headers = [
('Content-type','text/plain'), ('Content-type','text/plain'),
('Transfer-Encoding', "chunked"), ('Transfer-Encoding', "chunked"),
] ]
print 'request received' sys.stdout.write('request received')
sys.stdout.flush()
start_response(status, response_headers) start_response(status, response_headers)
return TestIter() return TestIter()

8
examples/multiapp.py
View File

@ -1,6 +1,6 @@
# -*- coding: utf-8 - # -*- coding: utf-8 -
# #
# This file is part of gunicorn released under the MIT license. # This file is part of gunicorn released under the MIT license.
# See the NOTICE for more information. # See the NOTICE for more information.
# #
# Run this application with: # Run this application with:
@ -17,7 +17,7 @@
try: try:
from routes import Mapper from routes import Mapper
except: except:
print "This example requires Routes to be installed" print("This example requires Routes to be installed")
# Obviously you'd import your app callables # Obviously you'd import your app callables
# from different places... # from different places...
@ -38,7 +38,7 @@ class Application(object):
return match[0]['app'](environ, start_response) return match[0]['app'](environ, start_response)
def error404(self, environ, start_response): def error404(self, environ, start_response):
html = """\ html = b"""\
<html> <html>
<head> <head>
<title>404 - Not Found</title> <title>404 - Not Found</title>
@ -55,4 +55,4 @@ class Application(object):
start_response('404 Not Found', headers) start_response('404 Not Found', headers)
return [html] return [html]
app = Application() app = Application()

8
examples/multidomainapp.py
View File

@ -1,6 +1,6 @@
# -*- coding: utf-8 - # -*- coding: utf-8 -
# #
# This file is part of gunicorn released under the MIT license. # This file is part of gunicorn released under the MIT license.
# See the NOTICE for more information. # See the NOTICE for more information.
import re import re
@ -20,15 +20,15 @@ class SubDomainApp:
return app(environ, start_response) return app(environ, start_response)
else: else:
start_response("404 Not Found", []) start_response("404 Not Found", [])
return [""] return [b""]
def hello(environ, start_response): def hello(environ, start_response):
start_response("200 OK", [("Content-Type", "text/plain")]) start_response("200 OK", [("Content-Type", "text/plain")])
return ["Hello, world\n"] return [b"Hello, world\n"]
def bye(environ, start_response): def bye(environ, start_response):
start_response("200 OK", [("Content-Type", "text/plain")]) start_response("200 OK", [("Content-Type", "text/plain")])
return ["Goodbye!\n"] return [b"Goodbye!\n"]
app = SubDomainApp([ app = SubDomainApp([
("localhost", hello), ("localhost", hello),

11
examples/slowclient.py
View File

@ -1,20 +1,21 @@
# -*- coding: utf-8 - # -*- coding: utf-8 -
# #
# This file is part of gunicorn released under the MIT license. # This file is part of gunicorn released under the MIT license.
# See the NOTICE for more information. # See the NOTICE for more information.
import sys
import time import time
def app(environ, start_response): def app(environ, start_response):
"""Application which cooperatively pauses 10 seconds before responding""" """Application which cooperatively pauses 10 seconds before responding"""
data = 'Hello, World!\n' data = b'Hello, World!\n'
status = '200 OK' status = '200 OK'
response_headers = [ response_headers = [
('Content-type','text/plain'), ('Content-type','text/plain'),
('Content-Length', str(len(data))) ] ('Content-Length', str(len(data))) ]
print 'request received, pausing 10 seconds' sys.stdout.write('request received, pausing 10 seconds')
sys.stdout.flush()
time.sleep(10) time.sleep(10)
start_response(status, response_headers) start_response(status, response_headers)
return iter([data]) return iter([data])

8
examples/test.py
View File

@ -12,14 +12,14 @@ from gunicorn import __version__
#@validator #@validator
def app(environ, start_response): def app(environ, start_response):
"""Simplest possible application object""" """Simplest possible application object"""
data = 'Hello, World!\n' data = b'Hello, World!\n'
status = '200 OK' status = '200 OK'
# print("print to stdout in test app")
# sys.stderr.write("stderr, print to stderr in test app\n")
response_headers = [ response_headers = [
('Content-type','text/plain'), ('Content-type','text/plain'),
('Content-Length', str(len(data))), ('Content-Length', str(len(data))),
('X-Gunicorn-Version', __version__) ('X-Gunicorn-Version', __version__),
("Test", "test тест"),
] ]
start_response(status, response_headers) start_response(status, response_headers)
return iter([data]) return iter([data])

4
gunicorn/__init__.py
View File

@ -3,6 +3,6 @@
# This file is part of gunicorn released under the MIT license. # This file is part of gunicorn released under the MIT license.
# See the NOTICE for more information. # See the NOTICE for more information.
version_info = (0, 15, 0) version_info = (0, 16, 0)
__version__ = ".".join(map(str, version_info)) __version__ = ".".join([str(v) for v in version_info])
SERVER_SOFTWARE = "gunicorn/%s" % __version__ SERVER_SOFTWARE = "gunicorn/%s" % __version__

23
gunicorn/app/base.py
View File

@ -3,17 +3,15 @@
# This file is part of gunicorn released under the MIT license. # This file is part of gunicorn released under the MIT license.
# See the NOTICE for more information. # See the NOTICE for more information.
import errno
import os import os
import sys import sys
import traceback import traceback
from gunicorn.glogging import Logger
from gunicorn import util from gunicorn import util
from gunicorn.arbiter import Arbiter from gunicorn.arbiter import Arbiter
from gunicorn.config import Config from gunicorn.config import Config
from gunicorn import debug from gunicorn import debug
from gunicorn.six import execfile_
class Application(object): class Application(object):
"""\ """\
@ -31,7 +29,7 @@ class Application(object):
def do_load_config(self): def do_load_config(self):
try: try:
self.load_config() self.load_config()
except Exception, e: except Exception as e:
sys.stderr.write("\nError: %s\n" % str(e)) sys.stderr.write("\nError: %s\n" % str(e))
sys.stderr.flush() sys.stderr.flush()
sys.exit(1) sys.exit(1)
@ -62,9 +60,9 @@ class Application(object):
"__package__": None "__package__": None
} }
try: try:
execfile(opts.config, cfg, cfg) execfile_(opts.config, cfg, cfg)
except Exception: except Exception:
print "Failed to read config file: %s" % opts.config print("Failed to read config file: %s" % opts.config)
traceback.print_exc() traceback.print_exc()
sys.exit(1) sys.exit(1)
@ -104,15 +102,12 @@ class Application(object):
def run(self): def run(self):
if self.cfg.check_config: if self.cfg.check_config:
try: try:
self.load() self.load()
except: except:
sys.stderr.write("\nError while loading the application:\n\n") sys.stderr.write("\nError while loading the application:\n\n")
traceback.print_exc() traceback.print_exc()
sys.stderr.flush() sys.stderr.flush()
sys.exit(1) sys.exit(1)
sys.exit(0) sys.exit(0)
if self.cfg.spew: if self.cfg.spew:
@ -120,9 +115,17 @@ class Application(object):
if self.cfg.daemon: if self.cfg.daemon:
util.daemonize() util.daemonize()
# set python paths
if self.cfg.pythonpath and self.cfg.pythonpath is not None:
paths = self.cfg.pythonpath.split(",")
for path in paths:
pythonpath = os.path.abspath(self.cfg.pythonpath)
if pythonpath not in sys.path:
sys.path.insert(0, pythonpath)
try: try:
Arbiter(self).run() Arbiter(self).run()
except RuntimeError, e: except RuntimeError as e:
sys.stderr.write("\nError: %s\n\n" % e) sys.stderr.write("\nError: %s\n\n" % e)
sys.stderr.flush() sys.stderr.flush()
sys.exit(1) sys.exit(1)

10
gunicorn/app/django_wsgi.py
View File

@ -10,10 +10,12 @@ import re
import sys import sys
import time import time
try: try:
from cStringIO import StringIO from io import StringIO
from imp import reload
except ImportError: except ImportError:
from StringIO import StringIO from StringIO import StringIO
from django.conf import settings from django.conf import settings
from django.core.management.validation import get_validation_errors from django.core.management.validation import get_validation_errors
from django.utils import translation from django.utils import translation
@ -72,10 +74,10 @@ def reload_django_settings():
app_mod = util.import_module(app[:-2]) app_mod = util.import_module(app[:-2])
appdir = os.path.dirname(app_mod.__file__) appdir = os.path.dirname(app_mod.__file__)
app_subdirs = os.listdir(appdir) app_subdirs = os.listdir(appdir)
app_subdirs.sort()
name_pattern = re.compile(r'[a-zA-Z]\w*') name_pattern = re.compile(r'[a-zA-Z]\w*')
for d in app_subdirs: for d in sorted(app_subdirs):
if name_pattern.match(d) and os.path.isdir(os.path.join(appdir, d)): if (name_pattern.match(d) and
os.path.isdir(os.path.join(appdir, d))):
new_installed_apps.append('%s.%s' % (app[:-2], d)) new_installed_apps.append('%s.%s' % (app[:-2], d))
else: else:
new_installed_apps.append(app) new_installed_apps.append(app)

15
gunicorn/app/djangoapp.py
View File

@ -44,12 +44,14 @@ def make_default_env(cfg):
os.environ['DJANGO_SETTINGS_MODULE'] = cfg.django_settings os.environ['DJANGO_SETTINGS_MODULE'] = cfg.django_settings
if cfg.pythonpath and cfg.pythonpath is not None: if cfg.pythonpath and cfg.pythonpath is not None:
pythonpath = os.path.abspath(cfg.pythonpath) paths = cfg.pythonpath.split(",")
if pythonpath not in sys.path: for path in paths:
sys.path.insert(0, pythonpath) pythonpath = os.path.abspath(cfg.pythonpath)
if pythonpath not in sys.path:
sys.path.insert(0, pythonpath)
try: try:
_ = os.environ['DJANGO_SETTINGS_MODULE'] os.environ['DJANGO_SETTINGS_MODULE']
except KeyError: except KeyError:
# not settings env set, try to build one. # not settings env set, try to build one.
project_path, settings_name = find_settings_module(os.getcwd()) project_path, settings_name = find_settings_module(os.getcwd())
@ -71,12 +73,15 @@ class DjangoApplication(Application):
def init(self, parser, opts, args): def init(self, parser, opts, args):
if args: if args:
if "." in args[0]: if ("." in args[0] and not (os.path.isfile(args[0])
or os.path.isdir(args[0]))):
self.cfg.set("django_settings", args[0]) self.cfg.set("django_settings", args[0])
else: else:
# not settings env set, try to build one. # not settings env set, try to build one.
project_path, settings_name = find_settings_module( project_path, settings_name = find_settings_module(
os.path.abspath(args[0])) os.path.abspath(args[0]))
if project_path not in sys.path:
sys.path.insert(0, project_path)
if not project_path: if not project_path:
raise RuntimeError("django project not found") raise RuntimeError("django project not found")

8
gunicorn/app/pasterapp.py
View File

@ -6,7 +6,11 @@
import os import os
import pkg_resources import pkg_resources
import sys import sys
import ConfigParser
try:
import configparser as ConfigParser
except ImportError:
import ConfigParser
from paste.deploy import loadapp, loadwsgi from paste.deploy import loadapp, loadwsgi
SERVER = loadwsgi.SERVER SERVER = loadwsgi.SERVER
@ -118,7 +122,7 @@ class PasterServerApplication(PasterBaseApplication):
for k, v in cfg.items(): for k, v in cfg.items():
if k.lower() in self.cfg.settings and v is not None: if k.lower() in self.cfg.settings and v is not None:
self.cfg.set(k.lower(), v) self.cfg.set(k.lower(), v)
except Exception, e: except Exception as e:
sys.stderr.write("\nConfig error: %s\n" % str(e)) sys.stderr.write("\nConfig error: %s\n" % str(e))
sys.stderr.flush() sys.stderr.flush()
sys.exit(1) sys.exit(1)

44
gunicorn/arbiter.py
View File

@ -41,10 +41,8 @@ class Arbiter(object):
# I love dynamic languages # I love dynamic languages
SIG_QUEUE = [] SIG_QUEUE = []
SIGNALS = map( SIGNALS = [getattr(signal, "SIG%s" % x) \
lambda x: getattr(signal, "SIG%s" % x), for x in "HUP QUIT INT TERM TTIN TTOU USR1 USR2 WINCH".split()]
"HUP QUIT INT TERM TTIN TTOU USR1 USR2 WINCH".split()
)
SIG_NAMES = dict( SIG_NAMES = dict(
(getattr(signal, name), name[3:].lower()) for name in dir(signal) (getattr(signal, name), name[3:].lower()) for name in dir(signal)
if name[:3] == "SIG" and name[3] != "_" if name[:3] == "SIG" and name[3] != "_"
@ -99,7 +97,10 @@ class Arbiter(object):
if self.cfg.debug: if self.cfg.debug:
self.log.debug("Current configuration:") self.log.debug("Current configuration:")
for config, value in sorted(self.cfg.settings.iteritems()):
for config, value in sorted(self.cfg.settings.items(),
key=lambda setting: setting[1]):
self.log.debug(" %s: %s", config, value.value) self.log.debug(" %s: %s", config, value.value)
if self.cfg.preload_app: if self.cfg.preload_app:
@ -135,13 +136,20 @@ class Arbiter(object):
Initialize master signal handling. Most of the signals Initialize master signal handling. Most of the signals
are queued. Child signals only wake up the master. are queued. Child signals only wake up the master.
""" """
# close old PIPE
if self.PIPE: if self.PIPE:
map(os.close, self.PIPE) [os.close(p) for p in self.PIPE]
# initialize the pipe
self.PIPE = pair = os.pipe() self.PIPE = pair = os.pipe()
map(util.set_non_blocking, pair) for p in pair:
map(util.close_on_exec, pair) util.set_non_blocking(p)
util.close_on_exec(p)
self.log.close_on_exec() self.log.close_on_exec()
map(lambda s: signal.signal(s, self.signal), self.SIGNALS)
# intialiatze all signals
[signal.signal(s, self.signal) for s in self.SIGNALS]
signal.signal(signal.SIGCHLD, self.handle_chld) signal.signal(signal.SIGCHLD, self.handle_chld)
def signal(self, sig, frame): def signal(self, sig, frame):
@ -181,7 +189,7 @@ class Arbiter(object):
self.halt() self.halt()
except KeyboardInterrupt: except KeyboardInterrupt:
self.halt() self.halt()
except HaltServer, inst: except HaltServer as inst:
self.halt(reason=inst.reason, exit_status=inst.exit_status) self.halt(reason=inst.reason, exit_status=inst.exit_status)
except SystemExit: except SystemExit:
raise raise
@ -270,8 +278,8 @@ class Arbiter(object):
Wake up the arbiter by writing to the PIPE Wake up the arbiter by writing to the PIPE
""" """
try: try:
os.write(self.PIPE[1], '.') os.write(self.PIPE[1], b'.')
except IOError, e: except IOError as e:
if e.errno not in [errno.EAGAIN, errno.EINTR]: if e.errno not in [errno.EAGAIN, errno.EINTR]:
raise raise
@ -296,10 +304,10 @@ class Arbiter(object):
return return
while os.read(self.PIPE[0], 1): while os.read(self.PIPE[0], 1):
pass pass
except select.error, e: except select.error as e:
if e[0] not in [errno.EAGAIN, errno.EINTR]: if e.args[0] not in [errno.EAGAIN, errno.EINTR]:
raise raise
except OSError, e: except OSError as e:
if e.errno not in [errno.EAGAIN, errno.EINTR]: if e.errno not in [errno.EAGAIN, errno.EINTR]:
raise raise
except KeyboardInterrupt: except KeyboardInterrupt:
@ -423,7 +431,7 @@ class Arbiter(object):
if not worker: if not worker:
continue continue
worker.tmp.close() worker.tmp.close()
except OSError, e: except OSError as e:
if e.errno == errno.ECHILD: if e.errno == errno.ECHILD:
pass pass
@ -436,7 +444,7 @@ class Arbiter(object):
self.spawn_workers() self.spawn_workers()
workers = self.WORKERS.items() workers = self.WORKERS.items()
workers.sort(key=lambda w: w[1].age) workers = sorted(workers, key=lambda w: w[1].age)
while len(workers) > self.num_workers: while len(workers) > self.num_workers:
(pid, _) = workers.pop(0) (pid, _) = workers.pop(0)
self.kill_worker(pid, signal.SIGQUIT) self.kill_worker(pid, signal.SIGQUIT)
@ -504,7 +512,7 @@ class Arbiter(object):
""" """
try: try:
os.kill(pid, sig) os.kill(pid, sig)
except OSError, e: except OSError as e:
if e.errno == errno.ESRCH: if e.errno == errno.ESRCH:
try: try:
worker = self.WORKERS.pop(pid) worker = self.WORKERS.pop(pid)

39
gunicorn/config.py
View File

@ -15,6 +15,7 @@ import types
from gunicorn import __version__ from gunicorn import __version__
from gunicorn.errors import ConfigError from gunicorn.errors import ConfigError
from gunicorn import util from gunicorn import util
from gunicorn.six import string_types, integer_types, bytes_to_str
KNOWN_SETTINGS = [] KNOWN_SETTINGS = []
@ -61,10 +62,12 @@ class Config(object):
} }
parser = optparse.OptionParser(**kwargs) parser = optparse.OptionParser(**kwargs)
keys = self.settings.keys() keys = list(self.settings)
def sorter(k): def sorter(k):
return (self.settings[k].section, self.settings[k].order) return (self.settings[k].section, self.settings[k].order)
keys.sort(key=sorter)
keys = sorted(self.settings, key=self.settings.__getitem__)
for k in keys: for k in keys:
self.settings[k].add_option(parser) self.settings[k].add_option(parser)
return parser return parser
@ -84,7 +87,7 @@ class Config(object):
@property @property
def address(self): def address(self):
bind = self.settings['bind'].get() bind = self.settings['bind'].get()
return util.parse_address(util.to_bytestring(bind)) return util.parse_address(bytes_to_str(bind))
@property @property
def uid(self): def uid(self):
@ -134,8 +137,6 @@ class SettingMeta(type):
setattr(cls, "short", desc.splitlines()[0]) setattr(cls, "short", desc.splitlines()[0])
class Setting(object): class Setting(object):
__metaclass__ = SettingMeta
name = None name = None
value = None value = None
section = None section = None
@ -178,10 +179,17 @@ class Setting(object):
assert callable(self.validator), "Invalid validator: %s" % self.name assert callable(self.validator), "Invalid validator: %s" % self.name
self.value = self.validator(val) self.value = self.validator(val)
def __lt__(self, other):
return (self.section == other.section and
self.order < other.order)
__cmp__ = __lt__
Setting = SettingMeta('Setting', (Setting,), {})
def validate_bool(val): def validate_bool(val):
if isinstance(val, types.BooleanType): if isinstance(val, bool):
return val return val
if not isinstance(val, basestring): if not isinstance(val, string_types):
raise TypeError("Invalid type for casting: %s" % val) raise TypeError("Invalid type for casting: %s" % val)
if val.lower().strip() == "true": if val.lower().strip() == "true":
return True return True
@ -196,7 +204,7 @@ def validate_dict(val):
return val return val
def validate_pos_int(val): def validate_pos_int(val):
if not isinstance(val, (types.IntType, types.LongType)): if not isinstance(val, integer_types):
val = int(val, 0) val = int(val, 0)
else: else:
# Booleans are ints! # Booleans are ints!
@ -208,7 +216,7 @@ def validate_pos_int(val):
def validate_string(val): def validate_string(val):
if val is None: if val is None:
return None return None
if not isinstance(val, basestring): if not isinstance(val, string_types):
raise TypeError("Not a string: %s" % val) raise TypeError("Not a string: %s" % val)
return val.strip() return val.strip()
@ -229,7 +237,7 @@ def validate_class(val):
def validate_callable(arity): def validate_callable(arity):
def _validate_callable(val): def _validate_callable(val):
if isinstance(val, basestring): if isinstance(val, string_types):
try: try:
mod_name, obj_name = val.rsplit(".", 1) mod_name, obj_name = val.rsplit(".", 1)
except ValueError: except ValueError:
@ -311,7 +319,12 @@ class Bind(Setting):
cli = ["-b", "--bind"] cli = ["-b", "--bind"]
meta = "ADDRESS" meta = "ADDRESS"
validator = validate_string validator = validate_string
default = "127.0.0.1:8000"
if 'PORT' in os.environ:
default = '0.0.0.0:{0}'.format(os.environ.get('PORT'))
else:
default = '127.0.0.1:8000'
desc = """\ desc = """\
The socket to bind. The socket to bind.
@ -863,9 +876,9 @@ class DjangoSettings(Setting):
DJANGO_SETTINGS_MODULE environment variable will be used. DJANGO_SETTINGS_MODULE environment variable will be used.
""" """
class DjangoPythonPath(Setting): class PythonPath(Setting):
name = "pythonpath" name = "pythonpath"
section = "Django" section = "Server Mechanics"
cli = ["--pythonpath"] cli = ["--pythonpath"]
meta = "STRING" meta = "STRING"
validator = validate_string validator = validate_string

4
gunicorn/debug.py
View File

@ -43,7 +43,7 @@ class Spew(object):
line = 'Unknown code named [%s]. VM instruction #%d' % ( line = 'Unknown code named [%s]. VM instruction #%d' % (
frame.f_code.co_name, frame.f_lasti) frame.f_code.co_name, frame.f_lasti)
if self.trace_names is None or name in self.trace_names: if self.trace_names is None or name in self.trace_names:
print '%s:%s: %s' % (name, lineno, line.rstrip()) print('%s:%s: %s' % (name, lineno, line.rstrip()))
if not self.show_values: if not self.show_values:
return self return self
details = [] details = []
@ -54,7 +54,7 @@ class Spew(object):
if tok in frame.f_locals: if tok in frame.f_locals:
details.append('%s=%r' % (tok, frame.f_locals[tok])) details.append('%s=%r' % (tok, frame.f_locals[tok]))
if details: if details:
print "\t%s" % ' '.join(details) print("\t%s" % ' '.join(details))
return self return self

4
gunicorn/errors.py
View File

@ -4,7 +4,7 @@
# See the NOTICE for more information. # See the NOTICE for more information.
class HaltServer(Exception): class HaltServer(BaseException):
def __init__(self, reason, exit_status=1): def __init__(self, reason, exit_status=1):
self.reason = reason self.reason = reason
self.exit_status = exit_status self.exit_status = exit_status
@ -12,5 +12,5 @@ class HaltServer(Exception):
def __str__(self): def __str__(self):
return "<HaltServer %r %d>" % (self.reason, self.exit_status) return "<HaltServer %r %d>" % (self.reason, self.exit_status)
class ConfigError(Exception): class ConfigError(BaseException):
""" Exception raised on config error """ """ Exception raised on config error """

25
gunicorn/glogging.py
View File

@ -6,17 +6,15 @@
import datetime import datetime
import logging import logging
logging.Logger.manager.emittedNoHandlerWarning = 1 logging.Logger.manager.emittedNoHandlerWarning = 1
from logging.config import fileConfig
import os import os
import sys import sys
import traceback import traceback
import threading import threading
try:
from logging.config import fileConfig
except ImportError:
from gunicorn.logging_config import fileConfig
from gunicorn import util from gunicorn import util
from gunicorn.six import string_types
CONFIG_DEFAULTS = dict( CONFIG_DEFAULTS = dict(
version = 1, version = 1,
@ -76,6 +74,16 @@ class LazyWriter(object):
self.lock.release() self.lock.release()
return self.fileobj return self.fileobj
def close(self):
if self.fileobj:
self.lock.acquire()
try:
if self.fileobj:
self.fileobj.close()
self.fileobj = None
finally:
self.lock.release()
def write(self, text): def write(self, text):
fileobj = self.open() fileobj = self.open()
fileobj.write(text) fileobj.write(text)
@ -89,6 +97,9 @@ class LazyWriter(object):
def flush(self): def flush(self):
self.open().flush() self.open().flush()
def isatty(self):
return bool(self.fileobj and self.fileobj.isatty())
class SafeAtoms(dict): class SafeAtoms(dict):
def __init__(self, atoms): def __init__(self, atoms):
@ -179,7 +190,7 @@ class Logger(object):
self.error_log.exception(msg, *args) self.error_log.exception(msg, *args)
def log(self, lvl, msg, *args, **kwargs): def log(self, lvl, msg, *args, **kwargs):
if isinstance(lvl, basestring): if isinstance(lvl, string_types):
lvl = self.LOG_LEVELS.get(lvl.lower(), logging.INFO) lvl = self.LOG_LEVELS.get(lvl.lower(), logging.INFO)
self.error_log.log(lvl, msg, *args, **kwargs) self.error_log.log(lvl, msg, *args, **kwargs)
@ -238,6 +249,10 @@ class Logger(object):
def reopen_files(self): def reopen_files(self):
if self.cfg.errorlog != "-":
# Close stderr & stdout if they are redirected to error log file
sys.stderr.close()
sys.stdout.close()
for log in loggers(): for log in loggers():
for handler in log.handlers: for handler in log.handlers:
if isinstance(handler, logging.FileHandler): if isinstance(handler, logging.FileHandler):

86
gunicorn/http/body.py
View File

@ -5,55 +5,51 @@
import sys import sys
try: from gunicorn.http.errors import (NoMoreData, ChunkMissingTerminator,
from cStringIO import StringIO InvalidChunkSize)
except ImportError: from gunicorn import six
from StringIO import StringIO
from gunicorn.http.errors import NoMoreData, ChunkMissingTerminator, \
InvalidChunkSize
class ChunkedReader(object): class ChunkedReader(object):
def __init__(self, req, unreader): def __init__(self, req, unreader):
self.req = req self.req = req
self.parser = self.parse_chunked(unreader) self.parser = self.parse_chunked(unreader)
self.buf = StringIO() self.buf = six.BytesIO()
def read(self, size): def read(self, size):
if not isinstance(size, (int, long)): if not isinstance(size, six.integer_types):
raise TypeError("size must be an integral type") raise TypeError("size must be an integral type")
if size <= 0: if size <= 0:
raise ValueError("Size must be positive.") raise ValueError("Size must be positive.")
if size == 0: if size == 0:
return "" return b""
if self.parser: if self.parser:
while self.buf.tell() < size: while self.buf.tell() < size:
try: try:
self.buf.write(self.parser.next()) self.buf.write(six.next(self.parser))
except StopIteration: except StopIteration:
self.parser = None self.parser = None
break break
data = self.buf.getvalue() data = self.buf.getvalue()
ret, rest = data[:size], data[size:] ret, rest = data[:size], data[size:]
self.buf.truncate(0) self.buf = six.BytesIO()
self.buf.write(rest) self.buf.write(rest)
return ret return ret
def parse_trailers(self, unreader, data): def parse_trailers(self, unreader, data):
buf = StringIO() buf = six.BytesIO()
buf.write(data) buf.write(data)
idx = buf.getvalue().find("\r\n\r\n") idx = buf.getvalue().find(b"\r\n\r\n")
done = buf.getvalue()[:2] == "\r\n" done = buf.getvalue()[:2] == b"\r\n"
while idx < 0 and not done: while idx < 0 and not done:
self.get_data(unreader, buf) self.get_data(unreader, buf)
idx = buf.getvalue().find("\r\n\r\n") idx = buf.getvalue().find(b"\r\n\r\n")
done = buf.getvalue()[:2] == "\r\n" done = buf.getvalue()[:2] == b"\r\n"
if done: if done:
unreader.unread(buf.getvalue()[2:]) unreader.unread(buf.getvalue()[2:])
return "" return b""
self.req.trailers = self.req.parse_headers(buf.getvalue()[:idx]) self.req.trailers = self.req.parse_headers(buf.getvalue()[:idx])
unreader.unread(buf.getvalue()[idx+4:]) unreader.unread(buf.getvalue()[idx+4:])
@ -71,24 +67,24 @@ class ChunkedReader(object):
rest = rest[size:] rest = rest[size:]
while len(rest) < 2: while len(rest) < 2:
rest += unreader.read() rest += unreader.read()
if rest[:2] != '\r\n': if rest[:2] != b'\r\n':
raise ChunkMissingTerminator(rest[:2]) raise ChunkMissingTerminator(rest[:2])
(size, rest) = self.parse_chunk_size(unreader, data=rest[2:]) (size, rest) = self.parse_chunk_size(unreader, data=rest[2:])
def parse_chunk_size(self, unreader, data=None): def parse_chunk_size(self, unreader, data=None):
buf = StringIO() buf = six.BytesIO()
if data is not None: if data is not None:
buf.write(data) buf.write(data)
idx = buf.getvalue().find("\r\n") idx = buf.getvalue().find(b"\r\n")
while idx < 0: while idx < 0:
self.get_data(unreader, buf) self.get_data(unreader, buf)
idx = buf.getvalue().find("\r\n") idx = buf.getvalue().find(b"\r\n")
data = buf.getvalue() data = buf.getvalue()
line, rest_chunk = data[:idx], data[idx+2:] line, rest_chunk = data[:idx], data[idx+2:]
chunk_size = line.split(";", 1)[0].strip() chunk_size = line.split(b";", 1)[0].strip()
try: try:
chunk_size = int(chunk_size, 16) chunk_size = int(chunk_size, 16)
except ValueError: except ValueError:
@ -114,17 +110,17 @@ class LengthReader(object):
self.length = length self.length = length
def read(self, size): def read(self, size):
if not isinstance(size, (int, long)): if not isinstance(size, six.integer_types):
raise TypeError("size must be an integral type") raise TypeError("size must be an integral type")
size = min(self.length, size) size = min(self.length, size)
if size < 0: if size < 0:
raise ValueError("Size must be positive.") raise ValueError("Size must be positive.")
if size == 0: if size == 0:
return "" return b""
buf = StringIO() buf = six.BytesIO()
data = self.unreader.read() data = self.unreader.read()
while data: while data:
buf.write(data) buf.write(data)
@ -141,21 +137,21 @@ class LengthReader(object):
class EOFReader(object): class EOFReader(object):
def __init__(self, unreader): def __init__(self, unreader):
self.unreader = unreader self.unreader = unreader
self.buf = StringIO() self.buf = six.BytesIO()
self.finished = False self.finished = False
def read(self, size): def read(self, size):
if not isinstance(size, (int, long)): if not isinstance(size, six.integer_types):
raise TypeError("size must be an integral type") raise TypeError("size must be an integral type")
if size < 0: if size < 0:
raise ValueError("Size must be positive.") raise ValueError("Size must be positive.")
if size == 0: if size == 0:
return "" return b""
if self.finished: if self.finished:
data = self.buf.getvalue() data = self.buf.getvalue()
ret, rest = data[:size], data[size:] ret, rest = data[:size], data[size:]
self.buf.truncate(0) self.buf = six.BytesIO()
self.buf.write(rest) self.buf.write(rest)
return ret return ret
@ -171,42 +167,43 @@ class EOFReader(object):
data = self.buf.getvalue() data = self.buf.getvalue()
ret, rest = data[:size], data[size:] ret, rest = data[:size], data[size:]
self.buf.truncate(0) self.buf = six.BytesIO()
self.buf.write(rest) self.buf.write(rest)
return ret return ret
class Body(object): class Body(object):
def __init__(self, reader): def __init__(self, reader):
self.reader = reader self.reader = reader
self.buf = StringIO() self.buf = six.BytesIO()
def __iter__(self): def __iter__(self):
return self return self
def next(self): def __next__(self):
ret = self.readline() ret = self.readline()
if not ret: if not ret:
raise StopIteration() raise StopIteration()
return ret return ret
next = __next__
def getsize(self, size): def getsize(self, size):
if size is None: if size is None:
return sys.maxint return six.MAXSIZE
elif not isinstance(size, (int, long)): elif not isinstance(size, six.integer_types):
raise TypeError("size must be an integral type") raise TypeError("size must be an integral type")
elif size < 0: elif size < 0:
return sys.maxint return six.MAXSIZE
return size return size
def read(self, size=None): def read(self, size=None):
size = self.getsize(size) size = self.getsize(size)
if size == 0: if size == 0:
return "" return b""
if size < self.buf.tell(): if size < self.buf.tell():
data = self.buf.getvalue() data = self.buf.getvalue()
ret, rest = data[:size], data[size:] ret, rest = data[:size], data[size:]
self.buf.truncate(0) self.buf = six.BytesIO()
self.buf.write(rest) self.buf.write(rest)
return ret return ret
@ -218,23 +215,23 @@ class Body(object):
data = self.buf.getvalue() data = self.buf.getvalue()
ret, rest = data[:size], data[size:] ret, rest = data[:size], data[size:]
self.buf.truncate(0) self.buf = six.BytesIO()
self.buf.write(rest) self.buf.write(rest)
return ret return ret
def readline(self, size=None): def readline(self, size=None):
size = self.getsize(size) size = self.getsize(size)
if size == 0: if size == 0:
return "" return b""
line = self.buf.getvalue() line = self.buf.getvalue()
self.buf.truncate(0) self.buf = six.BytesIO()
if len(line) < size: if len(line) < size:
line += self.reader.read(size - len(line)) line += self.reader.read(size - len(line))
extra_buf_data = line[size:] extra_buf_data = line[size:]
line = line[:size] line = line[:size]
idx = line.find("\n") idx = line.find(b"\n")
if idx >= 0: if idx >= 0:
ret = line[:idx+1] ret = line[:idx+1]
self.buf.write(line[idx+1:]) self.buf.write(line[idx+1:])
@ -247,12 +244,11 @@ class Body(object):
ret = [] ret = []
data = self.read() data = self.read()
while len(data): while len(data):
pos = data.find("\n") pos = data.find(b"\n")
if pos < 0: if pos < 0:
ret.append(data) ret.append(data)
data = "" data = b""
else: else:
line, data = data[:pos+1], data[pos+1:] line, data = data[:pos+1], data[pos+1:]
ret.append(line) ret.append(line)
return ret return ret

1
gunicorn/http/errors.py
View File

@ -9,6 +9,7 @@ class ParseException(Exception):
class NoMoreData(IOError): class NoMoreData(IOError):
def __init__(self, buf=None): def __init__(self, buf=None):
self.buf = buf self.buf = buf
def __str__(self): def __str__(self):
return "No more data after: %r" % self.buf return "No more data after: %r" % self.buf

38
gunicorn/http/message.py
View File

@ -4,21 +4,16 @@
# See the NOTICE for more information. # See the NOTICE for more information.
import re import re
import urlparse
import socket import socket
from errno import ENOTCONN from errno import ENOTCONN
try:
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO
from gunicorn.http.unreader import SocketUnreader from gunicorn.http.unreader import SocketUnreader
from gunicorn.http.body import ChunkedReader, LengthReader, EOFReader, Body from gunicorn.http.body import ChunkedReader, LengthReader, EOFReader, Body
from gunicorn.http.errors import InvalidHeader, InvalidHeaderName, NoMoreData, \ from gunicorn.http.errors import InvalidHeader, InvalidHeaderName, NoMoreData, \
InvalidRequestLine, InvalidRequestMethod, InvalidHTTPVersion, \ InvalidRequestLine, InvalidRequestMethod, InvalidHTTPVersion, \
LimitRequestLine, LimitRequestHeaders LimitRequestLine, LimitRequestHeaders
from gunicorn.http.errors import InvalidProxyLine, ForbiddenProxyRequest from gunicorn.http.errors import InvalidProxyLine, ForbiddenProxyRequest
from gunicorn.six import BytesIO, urlsplit, bytes_to_str
MAX_REQUEST_LINE = 8190 MAX_REQUEST_LINE = 8190
MAX_HEADERS = 32768 MAX_HEADERS = 32768
@ -61,7 +56,7 @@ class Message(object):
headers = [] headers = []
# Split lines on \r\n keeping the \r\n on each line # Split lines on \r\n keeping the \r\n on each line
lines = [line + "\r\n" for line in data.split("\r\n")] lines = [bytes_to_str(line) + "\r\n" for line in data.split(b"\r\n")]
# Parse headers into key/value pairs paying attention # Parse headers into key/value pairs paying attention
# to continuation lines. # to continuation lines.
@ -153,7 +148,6 @@ class Request(Message):
self.req_number = req_number self.req_number = req_number
self.proxy_protocol_info = None self.proxy_protocol_info = None
super(Request, self).__init__(cfg, unreader) super(Request, self).__init__(cfg, unreader)
@ -166,31 +160,31 @@ class Request(Message):
buf.write(data) buf.write(data)
def parse(self, unreader): def parse(self, unreader):
buf = StringIO() buf = BytesIO()
self.get_data(unreader, buf, stop=True) self.get_data(unreader, buf, stop=True)
# get request line # get request line
line, rbuf = self.read_line(unreader, buf, self.limit_request_line) line, rbuf = self.read_line(unreader, buf, self.limit_request_line)
# proxy protocol # proxy protocol
if self.proxy_protocol(line): if self.proxy_protocol(bytes_to_str(line)):
# get next request line # get next request line
buf = StringIO() buf = BytesIO()
buf.write(rbuf) buf.write(rbuf)
line, rbuf = self.read_line(unreader, buf, self.limit_request_line) line, rbuf = self.read_line(unreader, buf, self.limit_request_line)
self.parse_request_line(line) self.parse_request_line(bytes_to_str(line))
buf = StringIO() buf = BytesIO()
buf.write(rbuf) buf.write(rbuf)
# Headers # Headers
data = buf.getvalue() data = buf.getvalue()
idx = data.find("\r\n\r\n") idx = data.find(b"\r\n\r\n")
done = data[:2] == "\r\n" done = data[:2] == b"\r\n"
while True: while True:
idx = data.find("\r\n\r\n") idx = data.find(b"\r\n\r\n")
done = data[:2] == "\r\n" done = data[:2] == b"\r\n"
if idx < 0 and not done: if idx < 0 and not done:
self.get_data(unreader, buf) self.get_data(unreader, buf)
@ -202,19 +196,19 @@ class Request(Message):
if done: if done:
self.unreader.unread(data[2:]) self.unreader.unread(data[2:])
return "" return b""
self.headers = self.parse_headers(data[:idx]) self.headers = self.parse_headers(data[:idx])
ret = data[idx+4:] ret = data[idx+4:]
buf = StringIO() buf = BytesIO()
return ret return ret
def read_line(self, unreader, buf, limit=0): def read_line(self, unreader, buf, limit=0):
data = buf.getvalue() data = buf.getvalue()
while True: while True:
idx = data.find("\r\n") idx = data.find(b"\r\n")
if idx >= 0: if idx >= 0:
# check if the request line is too large # check if the request line is too large
if idx > limit > 0: if idx > limit > 0:
@ -256,7 +250,7 @@ class Request(Message):
try: try:
remote_host = self.unreader.sock.getpeername()[0] remote_host = self.unreader.sock.getpeername()[0]
except socket.error as e: except socket.error as e:
if e[0] == ENOTCONN: if e.args[0] == ENOTCONN:
raise ForbiddenProxyRequest("UNKNOW") raise ForbiddenProxyRequest("UNKNOW")
raise raise
if remote_host not in self.cfg.proxy_allow_ips: if remote_host not in self.cfg.proxy_allow_ips:
@ -328,7 +322,7 @@ class Request(Message):
else: else:
self.uri = bits[1] self.uri = bits[1]
parts = urlparse.urlsplit(self.uri) parts = urlsplit(self.uri)
self.path = parts.path or "" self.path = parts.path or ""
self.query = parts.query or "" self.query = parts.query or ""
self.fragment = parts.fragment or "" self.fragment = parts.fragment or ""

5
gunicorn/http/parser.py
View File

@ -22,7 +22,7 @@ class Parser(object):
def __iter__(self): def __iter__(self):
return self return self
def next(self): def __next__(self):
# Stop if HTTP dictates a stop. # Stop if HTTP dictates a stop.
if self.mesg and self.mesg.should_close(): if self.mesg and self.mesg.should_close():
raise StopIteration() raise StopIteration()
@ -33,6 +33,7 @@ class Parser(object):
while data: while data:
data = self.mesg.body.read(8192) data = self.mesg.body.read(8192)
# Parse the next request # Parse the next request
self.req_count += 1 self.req_count += 1
self.mesg = self.mesg_class(self.cfg, self.unreader, self.req_count) self.mesg = self.mesg_class(self.cfg, self.unreader, self.req_count)
@ -40,6 +41,8 @@ class Parser(object):
raise StopIteration() raise StopIteration()
return self.mesg return self.mesg
next = __next__
class RequestParser(Parser): class RequestParser(Parser):
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
super(RequestParser, self).__init__(Request, *args, **kwargs) super(RequestParser, self).__init__(Request, *args, **kwargs)

34
gunicorn/http/unreader.py
View File

@ -5,47 +5,47 @@
import os import os
try: from gunicorn import six
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO
# Classes that can undo reading data from # Classes that can undo reading data from
# a given type of data source. # a given type of data source.
class Unreader(object): class Unreader(object):
def __init__(self): def __init__(self):
self.buf = StringIO() self.buf = six.BytesIO()
def chunk(self): def chunk(self):
raise NotImplementedError() raise NotImplementedError()
def read(self, size=None): def read(self, size=None):
if size is not None and not isinstance(size, (int, long)): if size is not None and not isinstance(size, six.integer_types):
raise TypeError("size parameter must be an int or long.") raise TypeError("size parameter must be an int or long.")
if size == 0:
return "" if size is not None:
if size < 0: if size == 0:
size = None return b""
if size < 0:
size = None
self.buf.seek(0, os.SEEK_END) self.buf.seek(0, os.SEEK_END)
if size is None and self.buf.tell(): if size is None and self.buf.tell():
ret = self.buf.getvalue() ret = self.buf.getvalue()
self.buf.truncate(0) self.buf = six.BytesIO()
return ret return ret
if size is None: if size is None:
return self.chunk() d = self.chunk()
return d
while self.buf.tell() < size: while self.buf.tell() < size:
chunk = self.chunk() chunk = self.chunk()
if not len(chunk): if not len(chunk):
ret = self.buf.getvalue() ret = self.buf.getvalue()
self.buf.truncate(0) self.buf = six.BytesIO()
return ret return ret
self.buf.write(chunk) self.buf.write(chunk)
data = self.buf.getvalue() data = self.buf.getvalue()
self.buf.truncate(0) self.buf = six.BytesIO()
self.buf.write(data[size:]) self.buf.write(data[size:])
return data[:size] return data[:size]
@ -69,9 +69,9 @@ class IterUnreader(Unreader):
def chunk(self): def chunk(self):
if not self.iter: if not self.iter:
return "" return b""
try: try:
return self.iter.next() return six.next(self.iter)
except StopIteration: except StopIteration:
self.iter = None self.iter = None
return "" return b""

38
gunicorn/http/wsgi.py
View File

@ -7,8 +7,9 @@ import logging
import os import os
import re import re
import sys import sys
from urllib import unquote
from gunicorn.six import (unquote, string_types, binary_type, reraise,
text_type)
from gunicorn import SERVER_SOFTWARE from gunicorn import SERVER_SOFTWARE
import gunicorn.util as util import gunicorn.util as util
@ -54,7 +55,7 @@ def default_environ(req, sock, cfg):
"REQUEST_METHOD": req.method, "REQUEST_METHOD": req.method,
"QUERY_STRING": req.query, "QUERY_STRING": req.query,
"RAW_URI": req.uri, "RAW_URI": req.uri,
"SERVER_PROTOCOL": "HTTP/%s" % ".".join(map(str, req.version)) "SERVER_PROTOCOL": "HTTP/%s" % ".".join([str(v) for v in req.version])
} }
def proxy_environ(req): def proxy_environ(req):
@ -118,7 +119,7 @@ def create(req, sock, client, server, cfg):
environ['wsgi.url_scheme'] = url_scheme environ['wsgi.url_scheme'] = url_scheme
if isinstance(forward, basestring): if isinstance(forward, string_types):
# we only took the last one # we only took the last one
# http://en.wikipedia.org/wiki/X-Forwarded-For # http://en.wikipedia.org/wiki/X-Forwarded-For
if forward.find(",") >= 0: if forward.find(",") >= 0:
@ -145,7 +146,7 @@ def create(req, sock, client, server, cfg):
environ['REMOTE_ADDR'] = remote[0] environ['REMOTE_ADDR'] = remote[0]
environ['REMOTE_PORT'] = str(remote[1]) environ['REMOTE_PORT'] = str(remote[1])
if isinstance(server, basestring): if isinstance(server, string_types):
server = server.split(":") server = server.split(":")
if len(server) == 1: if len(server) == 1:
if url_scheme == "http": if url_scheme == "http":
@ -196,7 +197,7 @@ class Response(object):
if exc_info: if exc_info:
try: try:
if self.status and self.headers_sent: if self.status and self.headers_sent:
raise exc_info[0], exc_info[1], exc_info[2] reraise(exc_info[0], exc_info[1], exc_info[2])
finally: finally:
exc_info = None exc_info = None
elif self.status is not None: elif self.status is not None:
@ -209,7 +210,9 @@ class Response(object):
def process_headers(self, headers): def process_headers(self, headers):
for name, value in headers: for name, value in headers:
assert isinstance(name, basestring), "%r is not a string" % name assert isinstance(name, string_types), "%r is not a string" % name
value = str(value).strip()
lname = name.lower().strip() lname = name.lower().strip()
if lname == "content-length": if lname == "content-length":
self.response_length = int(value) self.response_length = int(value)
@ -220,11 +223,11 @@ class Response(object):
self.upgrade = True self.upgrade = True
elif lname == "upgrade": elif lname == "upgrade":
if value.lower().strip() == "websocket": if value.lower().strip() == "websocket":
self.headers.append((name.strip(), str(value).strip())) self.headers.append((name.strip(), value))
# ignore hopbyhop headers # ignore hopbyhop headers
continue continue
self.headers.append((name.strip(), str(value).strip())) self.headers.append((name.strip(), value))
def is_chunked(self): def is_chunked(self):
@ -265,13 +268,19 @@ class Response(object):
if self.headers_sent: if self.headers_sent:
return return
tosend = self.default_headers() tosend = self.default_headers()
tosend.extend(["%s: %s\r\n" % (n, v) for n, v in self.headers]) tosend.extend(["%s: %s\r\n" % (k,v) for k, v in self.headers])
util.write(self.sock, "%s\r\n" % "".join(tosend))
header_str = "%s\r\n" % "".join(tosend)
util.write(self.sock, util.to_bytestring(header_str))
self.headers_sent = True self.headers_sent = True
def write(self, arg): def write(self, arg):
self.send_headers() self.send_headers()
assert isinstance(arg, basestring), "%r is not a string." % arg
if isinstance(arg, text_type):
arg = arg.encode('utf-8')
assert isinstance(arg, binary_type), "%r is not a byte." % arg
arglen = len(arg) arglen = len(arg)
tosend = arglen tosend = arglen
@ -329,12 +338,13 @@ class Response(object):
self.send_headers() self.send_headers()
if self.is_chunked(): if self.is_chunked():
self.sock.sendall("%X\r\n" % nbytes) chunk_size = "%X\r\n" % nbytes
self.sock.sendall(chunk_size.encode('utf-8'))
self.sendfile_all(fileno, self.sock.fileno(), fo_offset, nbytes) self.sendfile_all(fileno, self.sock.fileno(), fo_offset, nbytes)
if self.is_chunked(): if self.is_chunked():
self.sock.sendall("\r\n") self.sock.sendall(b"\r\n")
os.lseek(fileno, fd_offset, os.SEEK_SET) os.lseek(fileno, fd_offset, os.SEEK_SET)
else: else:
@ -345,4 +355,4 @@ class Response(object):
if not self.headers_sent: if not self.headers_sent:
self.send_headers() self.send_headers()
if self.chunked: if self.chunked:
util.write_chunk(self.sock, "") util.write_chunk(self.sock, b"")

346
gunicorn/logging_config.py
View File

@ -1,346 +0,0 @@
# -*- coding: utf-8 -
#
# This file is part of gunicorn released under the MIT license.
# See the NOTICE for more information.
#
# Copyright 2001-2005 by Vinay Sajip. All Rights Reserved.
#
"""
Configuration functions for the logging package for Python. The core package
is based on PEP 282 and comments thereto in comp.lang.python, and influenced
by Apache's log4j system.
Should work under Python versions >= 1.5.2, except that source line
information is not available unless 'sys._getframe()' is.
Copyright (C) 2001-2004 Vinay Sajip. All Rights Reserved.
To use, simply 'import logging' and log away!
"""
import sys, logging, logging.handlers, string, socket, struct, os, traceback, types
try:
import thread
import threading
except ImportError:
thread = None
from SocketServer import ThreadingTCPServer, StreamRequestHandler
DEFAULT_LOGGING_CONFIG_PORT = 9030
if sys.platform == "win32":
RESET_ERROR = 10054 #WSAECONNRESET
else:
RESET_ERROR = 104 #ECONNRESET
#
# The following code implements a socket listener for on-the-fly
# reconfiguration of logging.
#
# _listener holds the server object doing the listening
_listener = None
def fileConfig(fname, defaults=None):
"""
Read the logging configuration from a ConfigParser-format file.
This can be called several times from an application, allowing an end user
the ability to select from various pre-canned configurations (if the
developer provides a mechanism to present the choices and load the chosen
configuration).
In versions of ConfigParser which have the readfp method [typically
shipped in 2.x versions of Python], you can pass in a file-like object
rather than a filename, in which case the file-like object will be read
using readfp.
"""
import ConfigParser
cp = ConfigParser.ConfigParser(defaults)
if hasattr(cp, 'readfp') and hasattr(fname, 'readline'):
cp.readfp(fname)
else:
cp.read(fname)
formatters = _create_formatters(cp)
# critical section
logging._acquireLock()
try:
logging._handlers.clear()
if hasattr(logging, '_handlerList'):
del logging._handlerList[:]
# Handlers add themselves to logging._handlers
handlers = _install_handlers(cp, formatters)
_install_loggers(cp, handlers)
finally:
logging._releaseLock()
def _resolve(name):
"""Resolve a dotted name to a global object."""
name = string.split(name, '.')
used = name.pop(0)
found = __import__(used)
for n in name:
used = used + '.' + n
try:
found = getattr(found, n)
except AttributeError:
__import__(used)
found = getattr(found, n)
return found
def _create_formatters(cp):
"""Create and return formatters"""
flist = cp.get("formatters", "keys")
if not len(flist):
return {}
flist = string.split(flist, ",")
formatters = {}
for form in flist:
form = string.strip(form)
sectname = "formatter_%s" % form
opts = cp.options(sectname)
if "format" in opts:
fs = cp.get(sectname, "format", 1)
else:
fs = None
if "datefmt" in opts:
dfs = cp.get(sectname, "datefmt", 1)
else:
dfs = None
c = logging.Formatter
if "class" in opts:
class_name = cp.get(sectname, "class")
if class_name:
c = _resolve(class_name)
f = c(fs, dfs)
formatters[form] = f
return formatters
def _install_handlers(cp, formatters):
"""Install and return handlers"""
hlist = cp.get("handlers", "keys")
if not len(hlist):
return {}
hlist = string.split(hlist, ",")
handlers = {}
fixups = [] #for inter-handler references
for hand in hlist:
hand = string.strip(hand)
sectname = "handler_%s" % hand
klass = cp.get(sectname, "class")
opts = cp.options(sectname)
if "formatter" in opts:
fmt = cp.get(sectname, "formatter")
else:
fmt = ""
try:
klass = eval(klass, vars(logging))
except (AttributeError, NameError):
klass = _resolve(klass)
args = cp.get(sectname, "args")
args = eval(args, vars(logging))
h = apply(klass, args)
if "level" in opts:
level = cp.get(sectname, "level")
h.setLevel(logging._levelNames[level])
if len(fmt):
h.setFormatter(formatters[fmt])
#temporary hack for FileHandler and MemoryHandler.
if klass == logging.handlers.MemoryHandler:
if "target" in opts:
target = cp.get(sectname,"target")
else:
target = ""
if len(target): #the target handler may not be loaded yet, so keep for later...
fixups.append((h, target))
handlers[hand] = h
#now all handlers are loaded, fixup inter-handler references...
for h, t in fixups:
h.setTarget(handlers[t])
return handlers
def _install_loggers(cp, handlers):
"""Create and install loggers"""
# configure the root first
llist = cp.get("loggers", "keys")
llist = string.split(llist, ",")
llist = map(lambda x: string.strip(x), llist)
llist.remove("root")
sectname = "logger_root"
root = logging.root
log = root
opts = cp.options(sectname)
if "level" in opts:
level = cp.get(sectname, "level")
log.setLevel(logging._levelNames[level])
for h in root.handlers[:]:
root.removeHandler(h)
hlist = cp.get(sectname, "handlers")
if len(hlist):
hlist = string.split(hlist, ",")
for hand in hlist:
log.addHandler(handlers[string.strip(hand)])
#and now the others...
#we don't want to lose the existing loggers,
#since other threads may have pointers to them.
#existing is set to contain all existing loggers,
#and as we go through the new configuration we
#remove any which are configured. At the end,
#what's left in existing is the set of loggers
#which were in the previous configuration but
#which are not in the new configuration.
existing = root.manager.loggerDict.keys()
#now set up the new ones...
for log in llist:
sectname = "logger_%s" % log
qn = cp.get(sectname, "qualname")
opts = cp.options(sectname)
if "propagate" in opts:
propagate = cp.getint(sectname, "propagate")
else:
propagate = 1
logger = logging.getLogger(qn)
if qn in existing:
existing.remove(qn)
if "level" in opts:
level = cp.get(sectname, "level")
logger.setLevel(logging._levelNames[level])
for h in logger.handlers[:]:
logger.removeHandler(h)
logger.propagate = propagate
logger.disabled = 0
hlist = cp.get(sectname, "handlers")
if len(hlist):
hlist = string.split(hlist, ",")
for hand in hlist:
logger.addHandler(handlers[string.strip(hand)])
#Disable any old loggers. There's no point deleting
#them as other threads may continue to hold references
#and by disabling them, you stop them doing any logging.
for log in existing:
root.manager.loggerDict[log].disabled = 1
def listen(port=DEFAULT_LOGGING_CONFIG_PORT):
"""
Start up a socket server on the specified port, and listen for new
configurations.
These will be sent as a file suitable for processing by fileConfig().
Returns a Thread object on which you can call start() to start the server,
and which you can join() when appropriate. To stop the server, call
stopListening().
"""
if not thread:
raise NotImplementedError, "listen() needs threading to work"
class ConfigStreamHandler(StreamRequestHandler):
"""
Handler for a logging configuration request.
It expects a completely new logging configuration and uses fileConfig
to install it.
"""
def handle(self):
"""
Handle a request.
Each request is expected to be a 4-byte length, packed using
struct.pack(">L", n), followed by the config file.
Uses fileConfig() to do the grunt work.
"""
import tempfile
try:
conn = self.connection
chunk = conn.recv(4)
if len(chunk) == 4:
slen = struct.unpack(">L", chunk)[0]
chunk = self.connection.recv(slen)
while len(chunk) < slen:
chunk = chunk + conn.recv(slen - len(chunk))
#Apply new configuration. We'd like to be able to
#create a StringIO and pass that in, but unfortunately
#1.5.2 ConfigParser does not support reading file
#objects, only actual files. So we create a temporary
#file and remove it later.
file = tempfile.mktemp(".ini")
f = open(file, "w")
f.write(chunk)
f.close()
try:
fileConfig(file)
except (KeyboardInterrupt, SystemExit):
raise
except:
traceback.print_exc()
os.remove(file)
except socket.error, e:
if type(e.args) != types.TupleType:
raise
else:
errcode = e.args[0]
if errcode != RESET_ERROR:
raise
class ConfigSocketReceiver(ThreadingTCPServer):
"""
A simple TCP socket-based logging config receiver.
"""
allow_reuse_address = 1
def __init__(self, host='localhost', port=DEFAULT_LOGGING_CONFIG_PORT,
handler=None):
ThreadingTCPServer.__init__(self, (host, port), handler)
logging._acquireLock()
self.abort = 0
logging._releaseLock()
self.timeout = 1
def serve_until_stopped(self):
import select
abort = 0
while not abort:
rd, wr, ex = select.select([self.socket.fileno()],
[], [],
self.timeout)
if rd:
self.handle_request()
logging._acquireLock()
abort = self.abort
logging._releaseLock()
def serve(rcvr, hdlr, port):
server = rcvr(port=port, handler=hdlr)
global _listener
logging._acquireLock()
_listener = server
logging._releaseLock()
server.serve_until_stopped()
return threading.Thread(target=serve,
args=(ConfigSocketReceiver,
ConfigStreamHandler, port))
def stopListening():
"""
Stop the listening server which was created with a call to listen().
"""
global _listener
if _listener:
logging._acquireLock()
_listener.abort = 1
_listener = None
logging._releaseLock()

8
gunicorn/pidfile.py
View File

@ -76,11 +76,11 @@ class Pidfile(object):
try: try:
os.kill(wpid, 0) os.kill(wpid, 0)
return wpid return wpid
except OSError, e: except OSError as e:
if e[0] == errno.ESRCH: if e.args[0] == errno.ESRCH:
return return
raise raise
except IOError, e: except IOError as e:
if e[0] == errno.ENOENT: if e.args[0] == errno.ENOENT:
return return
raise raise

399
gunicorn/six.py Normal file
View File

@ -0,0 +1,399 @@
"""Utilities for writing code that runs on Python 2 and 3"""
import operator
import sys
import types
__author__ = "Benjamin Peterson <benjamin@python.org>"
__version__ = "1.2.0"
# True if we are running on Python 3.
PY3 = sys.version_info[0] == 3
if PY3:
string_types = str,
integer_types = int,
class_types = type,
text_type = str
binary_type = bytes
MAXSIZE = sys.maxsize
else:
string_types = basestring,
integer_types = (int, long)
class_types = (type, types.ClassType)
text_type = unicode
binary_type = str
if sys.platform == "java":
# Jython always uses 32 bits.
MAXSIZE = int((1 << 31) - 1)
else:
# It's possible to have sizeof(long) != sizeof(Py_ssize_t).
class X(object):
def __len__(self):
return 1 << 31
try:
len(X())
except OverflowError:
# 32-bit
MAXSIZE = int((1 << 31) - 1)
else:
# 64-bit
MAXSIZE = int((1 << 63) - 1)
del X
def _add_doc(func, doc):
"""Add documentation to a function."""
func.__doc__ = doc
def _import_module(name):
"""Import module, returning the module after the last dot."""
__import__(name)
return sys.modules[name]
class _LazyDescr(object):
def __init__(self, name):
self.name = name
def __get__(self, obj, tp):
result = self._resolve()
setattr(obj, self.name, result)
# This is a bit ugly, but it avoids running this again.
delattr(tp, self.name)
return result
class MovedModule(_LazyDescr):
def __init__(self, name, old, new=None):
super(MovedModule, self).__init__(name)
if PY3:
if new is None:
new = name
self.mod = new
else:
self.mod = old
def _resolve(self):
return _import_module(self.mod)
class MovedAttribute(_LazyDescr):
def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None):
super(MovedAttribute, self).__init__(name)
if PY3:
if new_mod is None:
new_mod = name
self.mod = new_mod
if new_attr is None:
if old_attr is None:
new_attr = name
else:
new_attr = old_attr
self.attr = new_attr
else:
self.mod = old_mod
if old_attr is None:
old_attr = name
self.attr = old_attr
def _resolve(self):
module = _import_module(self.mod)
return getattr(module, self.attr)
class _MovedItems(types.ModuleType):
"""Lazy loading of moved objects"""
_moved_attributes = [
MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"),
MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"),
MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"),
MovedAttribute("map", "itertools", "builtins", "imap", "map"),
MovedAttribute("reload_module", "__builtin__", "imp", "reload"),
MovedAttribute("reduce", "__builtin__", "functools"),
MovedAttribute("StringIO", "StringIO", "io"),
MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"),
MovedAttribute("zip", "itertools", "builtins", "izip", "zip"),
MovedModule("builtins", "__builtin__"),
MovedModule("configparser", "ConfigParser"),
MovedModule("copyreg", "copy_reg"),
MovedModule("http_cookiejar", "cookielib", "http.cookiejar"),
MovedModule("http_cookies", "Cookie", "http.cookies"),
MovedModule("html_entities", "htmlentitydefs", "html.entities"),
MovedModule("html_parser", "HTMLParser", "html.parser"),
MovedModule("http_client", "httplib", "http.client"),
MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"),
MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"),
MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"),
MovedModule("cPickle", "cPickle", "pickle"),
MovedModule("queue", "Queue"),
MovedModule("reprlib", "repr"),
MovedModule("socketserver", "SocketServer"),
MovedModule("tkinter", "Tkinter"),
MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"),
MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"),
MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"),
MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"),
MovedModule("tkinter_tix", "Tix", "tkinter.tix"),
MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"),
MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"),
MovedModule("tkinter_colorchooser", "tkColorChooser",
"tkinter.colorchooser"),
MovedModule("tkinter_commondialog", "tkCommonDialog",
"tkinter.commondialog"),
MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"),
MovedModule("tkinter_font", "tkFont", "tkinter.font"),
MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"),
MovedModule("tkinter_tksimpledialog", "tkSimpleDialog",
"tkinter.simpledialog"),
MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"),
MovedModule("winreg", "_winreg"),
]
for attr in _moved_attributes:
setattr(_MovedItems, attr.name, attr)
del attr
moves = sys.modules["gunicorn.six.moves"] = _MovedItems("moves")
def add_move(move):
"""Add an item to six.moves."""
setattr(_MovedItems, move.name, move)
def remove_move(name):
"""Remove item from six.moves."""
try:
delattr(_MovedItems, name)
except AttributeError:
try:
del moves.__dict__[name]
except KeyError:
raise AttributeError("no such move, %r" % (name,))
if PY3:
_meth_func = "__func__"
_meth_self = "__self__"
_func_code = "__code__"
_func_defaults = "__defaults__"
_iterkeys = "keys"
_itervalues = "values"
_iteritems = "items"
else:
_meth_func = "im_func"
_meth_self = "im_self"
_func_code = "func_code"
_func_defaults = "func_defaults"
_iterkeys = "iterkeys"
_itervalues = "itervalues"
_iteritems = "iteritems"
try:
advance_iterator = next
except NameError:
def advance_iterator(it):
return it.next()
next = advance_iterator
if PY3:
def get_unbound_function(unbound):
return unbound
Iterator = object
def callable(obj):
return any("__call__" in klass.__dict__ for klass in type(obj).__mro__)
else:
def get_unbound_function(unbound):
return unbound.im_func
class Iterator(object):
def next(self):
return type(self).__next__(self)
callable = callable
_add_doc(get_unbound_function,
"""Get the function out of a possibly unbound function""")
get_method_function = operator.attrgetter(_meth_func)
get_method_self = operator.attrgetter(_meth_self)
get_function_code = operator.attrgetter(_func_code)
get_function_defaults = operator.attrgetter(_func_defaults)
def iterkeys(d):
"""Return an iterator over the keys of a dictionary."""
return iter(getattr(d, _iterkeys)())
def itervalues(d):
"""Return an iterator over the values of a dictionary."""
return iter(getattr(d, _itervalues)())
def iteritems(d):
"""Return an iterator over the (key, value) pairs of a dictionary."""
return iter(getattr(d, _iteritems)())
if PY3:
def b(s):
return s.encode("latin-1")
def u(s):
return s
if sys.version_info[1] <= 1:
def int2byte(i):
return bytes((i,))
else:
# This is about 2x faster than the implementation above on 3.2+
int2byte = operator.methodcaller("to_bytes", 1, "big")
import io
StringIO = io.StringIO
BytesIO = io.BytesIO
else:
def b(s):
return s
def u(s):
return unicode(s, "unicode_escape")
int2byte = chr
import StringIO
StringIO = BytesIO = StringIO.StringIO
_add_doc(b, """Byte literal""")
_add_doc(u, """Text literal""")
if PY3:
import builtins
exec_ = getattr(builtins, "exec")
def reraise(tp, value, tb=None):
if value.__traceback__ is not tb:
raise value.with_traceback(tb)
raise value
print_ = getattr(builtins, "print")
def execfile_(fname, *args):
return exec_(compile(open(fname, 'rb').read(), fname, 'exec'), *args)
del builtins
else:
def exec_(code, globs=None, locs=None):
"""Execute code in a namespace."""
if globs is None:
frame = sys._getframe(1)
globs = frame.f_globals
if locs is None:
locs = frame.f_locals
del frame
elif locs is None:
locs = globs
exec("""exec code in globs, locs""")
exec_("""def reraise(tp, value, tb=None):
raise tp, value, tb
""")
execfile_ = execfile
def print_(*args, **kwargs):
"""The new-style print function."""
fp = kwargs.pop("file", sys.stdout)
if fp is None:
return
def write(data):
if not isinstance(data, basestring):
data = str(data)
fp.write(data)
want_unicode = False
sep = kwargs.pop("sep", None)
if sep is not None:
if isinstance(sep, unicode):
want_unicode = True
elif not isinstance(sep, str):
raise TypeError("sep must be None or a string")
end = kwargs.pop("end", None)
if end is not None:
if isinstance(end, unicode):
want_unicode = True
elif not isinstance(end, str):
raise TypeError("end must be None or a string")
if kwargs:
raise TypeError("invalid keyword arguments to print()")
if not want_unicode:
for arg in args:
if isinstance(arg, unicode):
want_unicode = True
break
if want_unicode:
newline = unicode("\n")
space = unicode(" ")
else:
newline = "\n"
space = " "
if sep is None:
sep = space
if end is None:
end = newline
for i, arg in enumerate(args):
if i:
write(sep)
write(arg)
write(end)
_add_doc(reraise, """Reraise an exception.""")
def with_metaclass(meta, base=object):
"""Create a base class with a metaclass."""
return meta("NewBase", (base,), {})
# specific to gunicorn
if PY3:
def bytes_to_str(b):
if isinstance(b, text_type):
return b
return str(b, 'latin1')
import urllib.parse
unquote = urllib.parse.unquote
urlsplit = urllib.parse.urlsplit
urlparse = urllib.parse.urlparse
else:
def bytes_to_str(s):
if isinstance(s, unicode):
return s.encode('utf-8')
return s
import urlparse as orig_urlparse
urlsplit = orig_urlparse.urlsplit
urlparse = orig_urlparse.urlparse
import urllib
unquote = urllib.unquote

16
gunicorn/sock.py
View File

@ -10,7 +10,7 @@ import sys
import time import time
from gunicorn import util from gunicorn import util
from gunicorn.six import string_types
class BaseSocket(object): class BaseSocket(object):
@ -44,7 +44,7 @@ class BaseSocket(object):
def close(self): def close(self):
try: try:
self.sock.close() self.sock.close()
except socket.error, e: except socket.error as e:
self.log.info("Error while closing socket %s", str(e)) self.log.info("Error while closing socket %s", str(e))
time.sleep(0.3) time.sleep(0.3)
del self.sock del self.sock
@ -108,7 +108,7 @@ def create_socket(conf, log):
sock_type = TCP6Socket sock_type = TCP6Socket
else: else:
sock_type = TCPSocket sock_type = TCPSocket
elif isinstance(addr, basestring): elif isinstance(addr, string_types):
sock_type = UnixSocket sock_type = UnixSocket
else: else:
raise TypeError("Unable to create socket from: %r" % addr) raise TypeError("Unable to create socket from: %r" % addr)
@ -117,8 +117,8 @@ def create_socket(conf, log):
fd = int(os.environ.pop('GUNICORN_FD')) fd = int(os.environ.pop('GUNICORN_FD'))
try: try:
return sock_type(conf, log, fd=fd) return sock_type(conf, log, fd=fd)
except socket.error, e: except socket.error as e:
if e[0] == errno.ENOTCONN: if e.args[0] == errno.ENOTCONN:
log.error("GUNICORN_FD should refer to an open socket.") log.error("GUNICORN_FD should refer to an open socket.")
else: else:
raise raise
@ -130,10 +130,10 @@ def create_socket(conf, log):
for i in range(5): for i in range(5):
try: try:
return sock_type(conf, log) return sock_type(conf, log)
except socket.error, e: except socket.error as e:
if e[0] == errno.EADDRINUSE: if e.args[0] == errno.EADDRINUSE:
log.error("Connection in use: %s", str(addr)) log.error("Connection in use: %s", str(addr))
if e[0] == errno.EADDRNOTAVAIL: if e.args[0] == errno.EADDRNOTAVAIL:
log.error("Invalid address: %s", str(addr)) log.error("Invalid address: %s", str(addr))
sys.exit(1) sys.exit(1)
if i < 5: if i < 5:

32
gunicorn/util.py
View File

@ -25,6 +25,7 @@ import textwrap
import time import time
import inspect import inspect
from gunicorn.six import text_type, string_types
MAXFD = 1024 MAXFD = 1024
if (hasattr(os, "devnull")): if (hasattr(os, "devnull")):
@ -74,7 +75,7 @@ except ImportError:
if not hasattr(package, 'rindex'): if not hasattr(package, 'rindex'):
raise ValueError("'package' not set to a string") raise ValueError("'package' not set to a string")
dot = len(package) dot = len(package)
for x in xrange(level, 1, -1): for x in range(level, 1, -1):
try: try:
dot = package.rindex('.', 0, dot) dot = package.rindex('.', 0, dot)
except ValueError: except ValueError:
@ -125,7 +126,7 @@ def load_class(uri, default="sync", section="gunicorn.workers"):
return pkg_resources.load_entry_point("gunicorn", return pkg_resources.load_entry_point("gunicorn",
section, uri) section, uri)
except ImportError, e: except ImportError as e:
raise RuntimeError("class uri invalid or not found: " + raise RuntimeError("class uri invalid or not found: " +
"[%s]" % str(e)) "[%s]" % str(e))
klass = components.pop(-1) klass = components.pop(-1)
@ -216,14 +217,17 @@ try:
except ImportError: except ImportError:
def closerange(fd_low, fd_high): def closerange(fd_low, fd_high):
# Iterate through and close all file descriptors. # Iterate through and close all file descriptors.
for fd in xrange(fd_low, fd_high): for fd in range(fd_low, fd_high):
try: try:
os.close(fd) os.close(fd)
except OSError: # ERROR, fd wasn't open to begin with (ignored) except OSError: # ERROR, fd wasn't open to begin with (ignored)
pass pass
def write_chunk(sock, data): def write_chunk(sock, data):
chunk = "".join(("%X\r\n" % len(data), data, "\r\n")) if isinstance(data, text_type):
data = data.encode('utf-8')
chunk_size = "%X\r\n" % len(data)
chunk = b"".join([chunk_size.encode('utf-8'), data, b"\r\n"])
sock.sendall(chunk) sock.sendall(chunk)
def write(sock, data, chunked=False): def write(sock, data, chunked=False):
@ -267,7 +271,7 @@ def write_error(sock, status_int, reason, mesg):
\r \r
%s %s
""") % (str(status_int), reason, len(html), html) """) % (str(status_int), reason, len(html), html)
write_nonblock(sock, http) write_nonblock(sock, http.encode('latin1'))
def normalize_name(name): def normalize_name(name):
return "-".join([w.lower().capitalize() for w in name.split("-")]) return "-".join([w.lower().capitalize() for w in name.split("-")])
@ -307,15 +311,6 @@ def http_date(timestamp=None):
hh, mm, ss) hh, mm, ss)
return s return s
def to_bytestring(s):
""" convert to bytestring an unicode """
if not isinstance(s, basestring):
return s
if isinstance(s, unicode):
return s.encode('utf-8')
else:
return s
def is_hoppish(header): def is_hoppish(header):
return header.lower().strip() in hop_headers return header.lower().strip() in hop_headers
@ -350,6 +345,13 @@ def seed():
def check_is_writeable(path): def check_is_writeable(path):
try: try:
f = open(path, 'a') f = open(path, 'a')
except IOError, e: except IOError as e:
raise RuntimeError("Error: '%s' isn't writable [%r]" % (path, e)) raise RuntimeError("Error: '%s' isn't writable [%r]" % (path, e))
f.close() f.close()
def to_bytestring(value):
"""Converts a string argument to a byte string"""
if isinstance(value, bytes):
return value
assert isinstance(value, text_type)
return value.encode("utf-8")

22
gunicorn/workers/async.py
View File

@ -13,6 +13,7 @@ import gunicorn.http as http
import gunicorn.http.wsgi as wsgi import gunicorn.http.wsgi as wsgi
import gunicorn.util as util import gunicorn.util as util
import gunicorn.workers.base as base import gunicorn.workers.base as base
from gunicorn import six
ALREADY_HANDLED = object() ALREADY_HANDLED = object()
@ -28,40 +29,37 @@ class AsyncWorker(base.Worker):
def handle(self, client, addr): def handle(self, client, addr):
req = None req = None
try: try:
client.settimeout(self.cfg.timeout)
parser = http.RequestParser(self.cfg, client) parser = http.RequestParser(self.cfg, client)
try: try:
if not self.cfg.keepalive: if not self.cfg.keepalive:
req = parser.next() req = six.next(parser)
self.handle_request(req, client, addr) self.handle_request(req, client, addr)
else: else:
# keepalive loop # keepalive loop
while True: while True:
req = None req = None
with self.timeout_ctx(): with self.timeout_ctx():
req = parser.next() req = six.next(parser)
if not req: if not req:
break break
self.handle_request(req, client, addr) self.handle_request(req, client, addr)
except http.errors.NoMoreData, e: except http.errors.NoMoreData as e:
self.log.debug("Ignored premature client disconnection. %s", e) self.log.debug("Ignored premature client disconnection. %s", e)
except StopIteration, e: except StopIteration as e:
self.log.debug("Closing connection. %s", e) self.log.debug("Closing connection. %s", e)
except socket.error: except socket.error:
raise # pass to next try-except level raise # pass to next try-except level
except Exception, e: except Exception as e:
self.handle_error(req, client, addr, e) self.handle_error(req, client, addr, e)
except socket.timeout as e: except socket.error as e:
self.handle_error(req, client, addr, e) if e.args[0] not in (errno.EPIPE, errno.ECONNRESET):
except socket.error, e:
if e[0] not in (errno.EPIPE, errno.ECONNRESET):
self.log.exception("Socket error processing request.") self.log.exception("Socket error processing request.")
else: else:
if e[0] == errno.ECONNRESET: if e.args[0] == errno.ECONNRESET:
self.log.debug("Ignoring connection reset") self.log.debug("Ignoring connection reset")
else: else:
self.log.debug("Ignoring EPIPE") self.log.debug("Ignoring EPIPE")
except Exception, e: except Exception as e:
self.handle_error(req, client, addr, e) self.handle_error(req, client, addr, e)
finally: finally:
util.close(client) util.close(client)

23
gunicorn/workers/base.py
View File

@ -8,7 +8,6 @@ import os
import signal import signal
import sys import sys
import traceback import traceback
import socket
from gunicorn import util from gunicorn import util
@ -18,13 +17,12 @@ InvalidRequestLine, InvalidRequestMethod, InvalidHTTPVersion, \
LimitRequestLine, LimitRequestHeaders LimitRequestLine, LimitRequestHeaders
from gunicorn.http.errors import InvalidProxyLine, ForbiddenProxyRequest from gunicorn.http.errors import InvalidProxyLine, ForbiddenProxyRequest
from gunicorn.http.wsgi import default_environ, Response from gunicorn.http.wsgi import default_environ, Response
from gunicorn.six import MAXSIZE
class Worker(object): class Worker(object):
SIGNALS = map( SIGNALS = [getattr(signal, "SIG%s" % x) \
lambda x: getattr(signal, "SIG%s" % x), for x in "HUP QUIT INT TERM USR1 USR2 WINCH CHLD".split()]
"HUP QUIT INT TERM USR1 USR2 WINCH CHLD".split()
)
PIPE = [] PIPE = []
@ -43,7 +41,7 @@ class Worker(object):
self.booted = False self.booted = False
self.nr = 0 self.nr = 0
self.max_requests = cfg.max_requests or sys.maxint self.max_requests = cfg.max_requests or MAXSIZE
self.alive = True self.alive = True
self.log = log self.log = log
self.debug = cfg.debug self.debug = cfg.debug
@ -87,8 +85,9 @@ class Worker(object):
# For waking ourselves up # For waking ourselves up
self.PIPE = os.pipe() self.PIPE = os.pipe()
map(util.set_non_blocking, self.PIPE) for p in self.PIPE:
map(util.close_on_exec, self.PIPE) util.set_non_blocking(p)
util.close_on_exec(p)
# Prevent fd inherientence # Prevent fd inherientence
util.close_on_exec(self.socket) util.close_on_exec(self.socket)
@ -105,7 +104,9 @@ class Worker(object):
self.run() self.run()
def init_signals(self): def init_signals(self):
map(lambda s: signal.signal(s, signal.SIG_DFL), self.SIGNALS) # reset signaling
[signal.signal(s, signal.SIG_DFL) for s in self.SIGNALS]
# init new signaling
signal.signal(signal.SIGQUIT, self.handle_quit) signal.signal(signal.SIGQUIT, self.handle_quit)
signal.signal(signal.SIGTERM, self.handle_exit) signal.signal(signal.SIGTERM, self.handle_exit)
signal.signal(signal.SIGINT, self.handle_exit) signal.signal(signal.SIGINT, self.handle_exit)
@ -164,10 +165,6 @@ class Worker(object):
error=str(exc), error=str(exc),
) )
) )
elif isinstance(exc, socket.timeout):
status_int = 408
reason = "Request Timeout"
mesg = "<p>The server timed out handling for the request</p>"
else: else:
self.log.exception("Error handling request") self.log.exception("Error handling request")

28
gunicorn/workers/sync.py
View File

@ -14,6 +14,7 @@ import gunicorn.http as http
import gunicorn.http.wsgi as wsgi import gunicorn.http.wsgi as wsgi
import gunicorn.util as util import gunicorn.util as util
import gunicorn.workers.base as base import gunicorn.workers.base as base
from gunicorn import six
class SyncWorker(base.Worker): class SyncWorker(base.Worker):
@ -40,8 +41,8 @@ class SyncWorker(base.Worker):
# process. # process.
continue continue
except socket.error, e: except socket.error as e:
if e[0] not in (errno.EAGAIN, errno.ECONNABORTED): if e.args[0] not in (errno.EAGAIN, errno.ECONNABORTED):
raise raise
# If our parent changed then we shut down. # If our parent changed then we shut down.
@ -54,10 +55,10 @@ class SyncWorker(base.Worker):
ret = select.select([self.socket], [], self.PIPE, self.timeout) ret = select.select([self.socket], [], self.PIPE, self.timeout)
if ret[0]: if ret[0]:
continue continue
except select.error, e: except select.error as e:
if e[0] == errno.EINTR: if e.args[0] == errno.EINTR:
continue continue
if e[0] == errno.EBADF: if e.args[0] == errno.EBADF:
if self.nr < 0: if self.nr < 0:
continue continue
else: else:
@ -67,22 +68,19 @@ class SyncWorker(base.Worker):
def handle(self, client, addr): def handle(self, client, addr):
req = None req = None
try: try:
client.settimeout(self.cfg.timeout)
parser = http.RequestParser(self.cfg, client) parser = http.RequestParser(self.cfg, client)
req = parser.next() req = six.next(parser)
self.handle_request(req, client, addr) self.handle_request(req, client, addr)
except http.errors.NoMoreData, e: except http.errors.NoMoreData as e:
self.log.debug("Ignored premature client disconnection. %s", e) self.log.debug("Ignored premature client disconnection. %s", e)
except StopIteration, e: except StopIteration as e:
self.log.debug("Closing connection. %s", e) self.log.debug("Closing connection. %s", e)
except socket.timeout as e: except socket.error as e:
self.handle_error(req, client, addr, e) if e.args[0] != errno.EPIPE:
except socket.error, e:
if e[0] != errno.EPIPE:
self.log.exception("Error processing request.") self.log.exception("Error processing request.")
else: else:
self.log.debug("Ignoring EPIPE") self.log.debug("Ignoring EPIPE")
except Exception, e: except Exception as e:
self.handle_error(req, client, addr, e) self.handle_error(req, client, addr, e)
finally: finally:
util.close(client) util.close(client)
@ -117,7 +115,7 @@ class SyncWorker(base.Worker):
respiter.close() respiter.close()
except socket.error: except socket.error:
raise raise
except Exception, e: except Exception as e:
# Only send back traceback in HTTP in debug mode. # Only send back traceback in HTTP in debug mode.
self.handle_error(req, client, addr, e) self.handle_error(req, client, addr, e)
return return

2
requirements_dev.txt Normal file
View File

@ -0,0 +1,2 @@
pytest
pytest-cov

3
setup.cfg
View File

@ -2,3 +2,6 @@
build-requires = python2-devel python-setuptools build-requires = python2-devel python-setuptools
requires = python-setuptools >= 0.6c6 python-ctypes requires = python-setuptools >= 0.6c6 python-ctypes
install_script = rpm/install install_script = rpm/install
[pytest]
norecursedirs = examples lib local src

85
setup.py
View File

@ -5,47 +5,83 @@
import os import os
from setuptools import setup, find_packages from setuptools import setup, find_packages, Command
import sys import sys
from gunicorn import __version__ from gunicorn import __version__
CLASSIFIERS = [
'Development Status :: 4 - Beta',
'Environment :: Other Environment',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Operating System :: MacOS :: MacOS X',
'Operating System :: POSIX',
'Programming Language :: Python',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.2',
'Programming Language :: Python :: 3.3',
'Topic :: Internet',
'Topic :: Utilities',
'Topic :: Software Development :: Libraries :: Python Modules',
'Topic :: Internet :: WWW/HTTP',
'Topic :: Internet :: WWW/HTTP :: WSGI',
'Topic :: Internet :: WWW/HTTP :: WSGI :: Server',
'Topic :: Internet :: WWW/HTTP :: Dynamic Content']
# read long description
with open(os.path.join(os.path.dirname(__file__), 'README.rst')) as f:
long_description = f.read()
# read dev requirements
fname = os.path.join(os.path.dirname(__file__), 'requirements_dev.txt')
with open(fname) as f:
tests_require = list(map(lambda l: l.strip(), f.readlines()))
class PyTest(Command):
user_options = [
("cov", None, "measure coverage")
]
def initialize_options(self):
self.cov = None
def finalize_options(self):
pass
def run(self):
import sys,subprocess
basecmd = [sys.executable, '-m', 'py.test']
if self.cov:
basecmd += ['--cov', 'gunicorn']
errno = subprocess.call(basecmd + ['tests'])
raise SystemExit(errno)
setup( setup(
name = 'gunicorn', name = 'gunicorn',
version = __version__, version = __version__,
description = 'WSGI HTTP Server for UNIX', description = 'WSGI HTTP Server for UNIX',
long_description = file( long_description = long_description,
os.path.join(
os.path.dirname(__file__),
'README.rst'
)
).read(),
author = 'Benoit Chesneau', author = 'Benoit Chesneau',
author_email = 'benoitc@e-engura.com', author_email = 'benoitc@e-engura.com',
license = 'MIT', license = 'MIT',
url = 'http://gunicorn.org', url = 'http://gunicorn.org',
classifiers = [ classifiers = CLASSIFIERS,
'Development Status :: 4 - Beta',
'Environment :: Other Environment',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Operating System :: MacOS :: MacOS X',
'Operating System :: POSIX',
'Programming Language :: Python',
'Topic :: Internet',
'Topic :: Utilities',
'Topic :: Software Development :: Libraries :: Python Modules',
'Topic :: Internet :: WWW/HTTP',
'Topic :: Internet :: WWW/HTTP :: WSGI',
'Topic :: Internet :: WWW/HTTP :: WSGI :: Server',
'Topic :: Internet :: WWW/HTTP :: Dynamic Content',
],
zip_safe = False, zip_safe = False,
packages = find_packages(exclude=['examples', 'tests']), packages = find_packages(exclude=['examples', 'tests']),
include_package_data = True, include_package_data = True,
tests_require = tests_require,
cmdclass = {'test': PyTest},
entry_points=""" entry_points="""
[console_scripts] [console_scripts]
@ -66,6 +102,5 @@ setup(
[paste.server_runner] [paste.server_runner]
main=gunicorn.app.pasterapp:paste_server main=gunicorn.app.pasterapp:paste_server
""", """
test_suite = 'nose.collector',
) )

61
tests/004-test-http-body.py
View File

@ -1,61 +0,0 @@
from StringIO import StringIO
import t
from gunicorn.http.body import Body
def assert_readline(payload, size, expected):
body = Body(StringIO(payload))
t.eq(body.readline(size), expected)
def test_readline_empty_body():
assert_readline("", None, "")
assert_readline("", 1, "")
def test_readline_zero_size():
assert_readline("abc", 0, "")
assert_readline("\n", 0, "")
def test_readline_new_line_before_size():
body = Body(StringIO("abc\ndef"))
t.eq(body.readline(4), "abc\n")
t.eq(body.readline(), "def")
def test_readline_new_line_after_size():
body = Body(StringIO("abc\ndef"))
t.eq(body.readline(2), "ab")
t.eq(body.readline(), "c\n")
def test_readline_no_new_line():
body = Body(StringIO("abcdef"))
t.eq(body.readline(), "abcdef")
body = Body(StringIO("abcdef"))
t.eq(body.readline(2), "ab")
t.eq(body.readline(2), "cd")
t.eq(body.readline(2), "ef")
def test_readline_buffer_loaded():
reader = StringIO("abc\ndef")
body = Body(reader)
body.read(1) # load internal buffer
reader.write("g\nhi")
reader.seek(7)
t.eq(body.readline(), "bc\n")
t.eq(body.readline(), "defg\n")
t.eq(body.readline(), "hi")
def test_readline_buffer_loaded_with_size():
body = Body(StringIO("abc\ndef"))
body.read(1) # load internal buffer
t.eq(body.readline(2), "bc")
t.eq(body.readline(2), "\n")
t.eq(body.readline(2), "de")
t.eq(body.readline(2), "f")

4
tests/requests/valid/001.py
View File

@ -7,5 +7,5 @@ request = {
("CONTENT-TYPE", "application/json"), ("CONTENT-TYPE", "application/json"),
("CONTENT-LENGTH", "14") ("CONTENT-LENGTH", "14")
], ],
"body": '{"nom": "nom"}' "body": b'{"nom": "nom"}'
} }

4
tests/requests/valid/002.py
View File

@ -7,5 +7,5 @@ request = {
("HOST", "0.0.0.0=5000"), ("HOST", "0.0.0.0=5000"),
("ACCEPT", "*/*") ("ACCEPT", "*/*")
], ],
"body": "" "body": b""
} }

4
tests/requests/valid/003.py
View File

@ -12,5 +12,5 @@ request = {
("KEEP-ALIVE", "300"), ("KEEP-ALIVE", "300"),
("CONNECTION", "keep-alive") ("CONNECTION", "keep-alive")
], ],
"body": "" "body": b""
} }

4
tests/requests/valid/004.py
View File

@ -5,5 +5,5 @@ request = {
"headers": [ "headers": [
("AAAAAAAAAAAAA", "++++++++++") ("AAAAAAAAAAAAA", "++++++++++")
], ],
"body": "" "body": b""
} }

4
tests/requests/valid/005.py
View File

@ -3,5 +3,5 @@ request = {
"uri": uri("/forums/1/topics/2375?page=1#posts-17408"), "uri": uri("/forums/1/topics/2375?page=1#posts-17408"),
"version": (1, 1), "version": (1, 1),
"headers": [], "headers": [],
"body": "" "body": b""
} }

4
tests/requests/valid/006.py
View File

@ -3,5 +3,5 @@ request = {
"uri": uri("/get_no_headers_no_body/world"), "uri": uri("/get_no_headers_no_body/world"),
"version": (1, 1), "version": (1, 1),
"headers": [], "headers": [],
"body": "" "body": b""
} }

4
tests/requests/valid/007.py
View File

@ -5,5 +5,5 @@ request = {
"headers": [ "headers": [
("ACCEPT", "*/*") ("ACCEPT", "*/*")
], ],
"body": "" "body": b""
} }

4
tests/requests/valid/008.py
View File

@ -5,5 +5,5 @@ request = {
"headers": [ "headers": [
("CONTENT-LENGTH", "5") ("CONTENT-LENGTH", "5")
], ],
"body": "HELLO" "body": b"HELLO"
} }

4
tests/requests/valid/009.py
View File

@ -7,5 +7,5 @@ request = {
("TRANSFER-ENCODING", "identity"), ("TRANSFER-ENCODING", "identity"),
("CONTENT-LENGTH", "5") ("CONTENT-LENGTH", "5")
], ],
"body": "World" "body": b"World"
} }

4
tests/requests/valid/010.py
View File

@ -5,5 +5,5 @@ request = {
"headers": [ "headers": [
("TRANSFER-ENCODING", "chunked"), ("TRANSFER-ENCODING", "chunked"),
], ],
"body": "all your base are belong to us" "body": b"all your base are belong to us"
} }

4
tests/requests/valid/011.py
View File

@ -5,5 +5,5 @@ request = {
"headers": [ "headers": [
("TRANSFER-ENCODING", "chunked") ("TRANSFER-ENCODING", "chunked")
], ],
"body": "hello world" "body": b"hello world"
} }

4
tests/requests/valid/012.py
View File

@ -5,9 +5,9 @@ request = {
"headers": [ "headers": [
("TRANSFER-ENCODING", "chunked") ("TRANSFER-ENCODING", "chunked")
], ],
"body": "hello world", "body": b"hello world",
"trailers": [ "trailers": [
("VARY", "*"), ("VARY", "*"),
("CONTENT-TYPE", "text/plain") ("CONTENT-TYPE", "text/plain")
] ]
} }

4
tests/requests/valid/013.py
View File

@ -5,5 +5,5 @@ request = {
"headers": [ "headers": [
("TRANSFER-ENCODING", "chunked") ("TRANSFER-ENCODING", "chunked")
], ],
"body": "hello world" "body": b"hello world"
} }

4
tests/requests/valid/014.py
View File

@ -3,5 +3,5 @@ request = {
"uri": uri('/with_"quotes"?foo="bar"'), "uri": uri('/with_"quotes"?foo="bar"'),
"version": (1, 1), "version": (1, 1),
"headers": [], "headers": [],
"body": "" "body": b""
} }

4
tests/requests/valid/015.py
View File

@ -7,5 +7,5 @@ request = {
("USER-AGENT", "ApacheBench/2.3"), ("USER-AGENT", "ApacheBench/2.3"),
("ACCEPT", "*/*") ("ACCEPT", "*/*")
], ],
"body": "" "body": b""
} }

4
tests/requests/valid/016.py
View File

@ -36,5 +36,5 @@ request = {
"uri": uri("/"), "uri": uri("/"),
"version": (1, 1), "version": (1, 1),
"headers": [("X-SSL-CERT", certificate)], "headers": [("X-SSL-CERT", certificate)],
"body": "" "body": b""
} }

2
tests/requests/valid/017.py
View File

@ -6,5 +6,5 @@ request = {
("IF-MATCH", "bazinga!"), ("IF-MATCH", "bazinga!"),
("IF-MATCH", "large-sound") ("IF-MATCH", "large-sound")
], ],
"body": "" "body": b""
} }

4
tests/requests/valid/018.py
View File

@ -3,7 +3,7 @@ req1 = {
"uri": uri("/first"), "uri": uri("/first"),
"version": (1, 1), "version": (1, 1),
"headers": [], "headers": [],
"body": "" "body": b""
} }
req2 = { req2 = {
@ -11,7 +11,7 @@ req2 = {
"uri": uri("/second"), "uri": uri("/second"),
"version": (1, 1), "version": (1, 1),
"headers": [], "headers": [],
"body": "" "body": b""
} }
request = [req1, req2] request = [req1, req2]

4
tests/requests/valid/019.py
View File

@ -3,5 +3,5 @@ request = {
"uri": uri("/first"), "uri": uri("/first"),
"version": (1, 0), "version": (1, 0),
"headers": [], "headers": [],
"body": "" "body": b""
} }

4
tests/requests/valid/020.py
View File

@ -3,5 +3,5 @@ request = {
"uri": uri("/first"), "uri": uri("/first"),
"version": (1, 0), "version": (1, 0),
"headers": [('CONTENT-LENGTH', '24')], "headers": [('CONTENT-LENGTH', '24')],
"body": "GET /second HTTP/1.1\r\n\r\n" "body": b"GET /second HTTP/1.1\r\n\r\n"
} }

4
tests/requests/valid/021.py
View File

@ -3,5 +3,5 @@ request = {
"uri": uri("/first"), "uri": uri("/first"),
"version": (1, 1), "version": (1, 1),
"headers": [("CONNECTION", "Close")], "headers": [("CONNECTION", "Close")],
"body": "" "body": b""
} }

6
tests/requests/valid/022.py
View File

@ -3,7 +3,7 @@ req1 = {
"uri": uri("/first"), "uri": uri("/first"),
"version": (1, 0), "version": (1, 0),
"headers": [("CONNECTION", "Keep-Alive")], "headers": [("CONNECTION", "Keep-Alive")],
"body": "" "body": b""
} }
req2 = { req2 = {
@ -11,7 +11,7 @@ req2 = {
"uri": uri("/second"), "uri": uri("/second"),
"version": (1, 1), "version": (1, 1),
"headers": [], "headers": [],
"body": "" "body": b""
} }
request = [req1, req2] request = [req1, req2]

6
tests/requests/valid/023.py
View File

@ -5,7 +5,7 @@ req1 = {
"headers": [ "headers": [
("TRANSFER-ENCODING", "chunked") ("TRANSFER-ENCODING", "chunked")
], ],
"body": "hello world" "body": b"hello world"
} }
req2 = { req2 = {
@ -13,7 +13,7 @@ req2 = {
"uri": uri("/second"), "uri": uri("/second"),
"version": (1, 1), "version": (1, 1),
"headers": [], "headers": [],
"body": "" "body": b""
} }
request = [req1, req2] request = [req1, req2]

4
tests/requests/valid/025.py
View File

@ -6,7 +6,7 @@ req1 = {
("CONTENT-LENGTH", "-1"), ("CONTENT-LENGTH", "-1"),
("TRANSFER-ENCODING", "chunked") ("TRANSFER-ENCODING", "chunked")
], ],
"body": "hello world" "body": b"hello world"
} }
req2 = { req2 = {
@ -17,7 +17,7 @@ req2 = {
("TRANSFER-ENCODING", "chunked"), ("TRANSFER-ENCODING", "chunked"),
("CONTENT-LENGTH", "-1"), ("CONTENT-LENGTH", "-1"),
], ],
"body": "hello world" "body": b"hello world"
} }
request = [req1, req2] request = [req1, req2]

2
tests/requests/valid/pp_01.py
View File

@ -12,5 +12,5 @@ request = {
("CONTENT-TYPE", "application/json"), ("CONTENT-TYPE", "application/json"),
("CONTENT-LENGTH", "14") ("CONTENT-LENGTH", "14")
], ],
"body": '{"nom": "nom"}' "body": b'{"nom": "nom"}'
} }

4
tests/requests/valid/pp_02.py
View File

@ -13,7 +13,7 @@ req1 = {
("CONTENT-LENGTH", "14"), ("CONTENT-LENGTH", "14"),
("CONNECTION", "keep-alive") ("CONNECTION", "keep-alive")
], ],
"body": '{"nom": "nom"}' "body": b'{"nom": "nom"}'
} }
@ -24,7 +24,7 @@ req2 = {
"headers": [ "headers": [
("TRANSFER-ENCODING", "chunked"), ("TRANSFER-ENCODING", "chunked"),
], ],
"body": "all your base are belong to us" "body": b"all your base are belong to us"
} }
request = [req1, req2] request = [req1, req2]

35
tests/t.py
View File

@ -1,43 +1,43 @@
# -*- coding: utf-8 - # -*- coding: utf-8 -
# Copyright 2009 Paul J. Davis <paul.joseph.davis@gmail.com> # Copyright 2009 Paul J. Davis <paul.joseph.davis@gmail.com>
# #
# This file is part of gunicorn released under the MIT license. # This file is part of gunicorn released under the MIT license.
# See the NOTICE for more information. # See the NOTICE for more information.
from __future__ import with_statement from __future__ import with_statement
import array import array
import os import os
from StringIO import StringIO
import tempfile import tempfile
dirname = os.path.dirname(__file__) dirname = os.path.dirname(__file__)
from gunicorn.http.parser import RequestParser from gunicorn.http.parser import RequestParser
from gunicorn.config import Config from gunicorn.config import Config
from gunicorn.six import BytesIO
def data_source(fname): def data_source(fname):
buf = StringIO() buf = BytesIO()
with open(fname) as handle: with open(fname) as handle:
for line in handle: for line in handle:
line = line.rstrip("\n").replace("\\r\\n", "\r\n") line = line.rstrip("\n").replace("\\r\\n", "\r\n")
buf.write(line) buf.write(line.encode('latin1'))
return buf return buf
class request(object): class request(object):
def __init__(self, name): def __init__(self, name):
self.fname = os.path.join(dirname, "requests", name) self.fname = os.path.join(dirname, "requests", name)
def __call__(self, func): def __call__(self, func):
def run(): def run():
src = data_source(self.fname) src = data_source(self.fname)
func(src, RequestParser(src)) func(src, RequestParser(src))
run.func_name = func.func_name run.func_name = func.func_name
return run return run
class FakeSocket(object): class FakeSocket(object):
def __init__(self, data): def __init__(self, data):
self.tmp = tempfile.TemporaryFile() self.tmp = tempfile.TemporaryFile()
if data: if data:
@ -47,32 +47,32 @@ class FakeSocket(object):
def fileno(self): def fileno(self):
return self.tmp.fileno() return self.tmp.fileno()
def len(self): def len(self):
return self.tmp.len return self.tmp.len
def recv(self, length=None): def recv(self, length=None):
return self.tmp.read() return self.tmp.read()
def recv_into(self, buf, length): def recv_into(self, buf, length):
tmp_buffer = self.tmp.read(length) tmp_buffer = self.tmp.read(length)
v = len(tmp_buffer) v = len(tmp_buffer)
for i, c in enumerate(tmp_buffer): for i, c in enumerate(tmp_buffer):
buf[i] = c buf[i] = c
return v return v
def send(self, data): def send(self, data):
self.tmp.write(data) self.tmp.write(data)
self.tmp.flush() self.tmp.flush()
def seek(self, offset, whence=0): def seek(self, offset, whence=0):
self.tmp.seek(offset, whence) self.tmp.seek(offset, whence)
class http_request(object): class http_request(object):
def __init__(self, name): def __init__(self, name):
self.fname = os.path.join(dirname, "requests", name) self.fname = os.path.join(dirname, "requests", name)
def __call__(self, func): def __call__(self, func):
def run(): def run():
fsock = FakeSocket(data_source(self.fname)) fsock = FakeSocket(data_source(self.fname))
@ -80,7 +80,7 @@ class http_request(object):
func(req) func(req)
run.func_name = func.func_name run.func_name = func.func_name
return run return run
def eq(a, b): def eq(a, b):
assert a == b, "%r != %r" % (a, b) assert a == b, "%r != %r" % (a, b)
@ -117,4 +117,3 @@ def raises(exctype, func, *args, **kwargs):
func_name = getattr(func, "func_name", "<builtin_function>") func_name = getattr(func, "func_name", "<builtin_function>")
raise AssertionError("Function %s did not raise %s" % ( raise AssertionError("Function %s did not raise %s" % (
func_name, exctype.__name__)) func_name, exctype.__name__))

4
tests/001-test-valid-requests.py → tests/test_001-valid-requests.py
View File

@ -1,6 +1,6 @@
# -*- coding: utf-8 - # -*- coding: utf-8 -
# #
# This file is part of gunicorn released under the MIT license. # This file is part of gunicorn released under the MIT license.
# See the NOTICE for more information. # See the NOTICE for more information.
import t import t
@ -9,6 +9,8 @@ import treq
import glob import glob
import os import os
dirname = os.path.dirname(__file__) dirname = os.path.dirname(__file__)
from py.test import skip
reqdir = os.path.join(dirname, "requests", "valid") reqdir = os.path.join(dirname, "requests", "valid")
def a_case(fname): def a_case(fname):

15
tests/002-test-invalid-requests.py → tests/test_002-invalid-requests.py
View File

@ -1,6 +1,6 @@
# -*- coding: utf-8 - # -*- coding: utf-8 -
# #
# This file is part of gunicorn released under the MIT license. # This file is part of gunicorn released under the MIT license.
# See the NOTICE for more information. # See the NOTICE for more information.
import t import t
@ -8,7 +8,8 @@ import treq
import glob import glob
import os import os
from nose.tools import raises
import pytest
dirname = os.path.dirname(__file__) dirname = os.path.dirname(__file__)
reqdir = os.path.join(dirname, "requests", "invalid") reqdir = os.path.join(dirname, "requests", "invalid")
@ -17,12 +18,12 @@ reqdir = os.path.join(dirname, "requests", "invalid")
def test_http_parser(): def test_http_parser():
for fname in glob.glob(os.path.join(reqdir, "*.http")): for fname in glob.glob(os.path.join(reqdir, "*.http")):
env = treq.load_py(os.path.splitext(fname)[0] + ".py") env = treq.load_py(os.path.splitext(fname)[0] + ".py")
expect = env['request'] expect = env['request']
cfg = env['cfg'] cfg = env['cfg']
req = treq.badrequest(fname) req = treq.badrequest(fname)
@raises(expect) with pytest.raises(expect):
def check(fname): def f(fname):
return req.check(cfg) return req.check(cfg)
f(fname)
yield check, fname # fname is pass so that we know which test failed

57
tests/003-test-config.py → tests/test_003-config.py
View File

@ -1,12 +1,10 @@
# -*- coding: utf-8 - # -*- coding: utf-8 -
# #
# This file is part of gunicorn released under the MIT license. # This file is part of gunicorn released under the MIT license.
# See the NOTICE for more information. # See the NOTICE for more information.
from __future__ import with_statement from __future__ import with_statement
from nose.plugins.skip import SkipTest
import t import t
import functools import functools
@ -23,14 +21,6 @@ def cfg_file():
def paster_ini(): def paster_ini():
return os.path.join(dirname, "..", "examples", "frameworks", "pylonstest", "nose.ini") return os.path.join(dirname, "..", "examples", "frameworks", "pylonstest", "nose.ini")
def PasterApp():
try:
from paste.deploy import loadapp, loadwsgi
except ImportError:
raise SkipTest()
from gunicorn.app.pasterapp import PasterApplication
return PasterApplication("no_usage")
class AltArgs(object): class AltArgs(object):
def __init__(self, args=None): def __init__(self, args=None):
self.args = args or [] self.args = args or []
@ -38,17 +28,17 @@ class AltArgs(object):
def __enter__(self): def __enter__(self):
sys.argv = self.args sys.argv = self.args
def __exit__(self, exc_type, exc_inst, traceback): def __exit__(self, exc_type, exc_inst, traceback):
sys.argv = self.orig sys.argv = self.orig
class NoConfigApp(Application): class NoConfigApp(Application):
def __init__(self): def __init__(self):
super(NoConfigApp, self).__init__("no_usage") super(NoConfigApp, self).__init__("no_usage")
def init(self, parser, opts, args): def init(self, parser, opts, args):
pass pass
def load(self): def load(self):
pass pass
@ -63,25 +53,25 @@ def test_property_access():
c = config.Config() c = config.Config()
for s in config.KNOWN_SETTINGS: for s in config.KNOWN_SETTINGS:
getattr(c, s.name) getattr(c, s.name)
# Class was loaded # Class was loaded
t.eq(c.worker_class, SyncWorker) t.eq(c.worker_class, SyncWorker)
# Debug affects workers # Debug affects workers
t.eq(c.workers, 1) t.eq(c.workers, 1)
c.set("workers", 3) c.set("workers", 3)
t.eq(c.workers, 3) t.eq(c.workers, 3)
# Address is parsed # Address is parsed
t.eq(c.address, ("127.0.0.1", 8000)) t.eq(c.address, ("127.0.0.1", 8000))
# User and group defaults # User and group defaults
t.eq(os.geteuid(), c.uid) t.eq(os.geteuid(), c.uid)
t.eq(os.getegid(), c.gid) t.eq(os.getegid(), c.gid)
# Proc name # Proc name
t.eq("gunicorn", c.proc_name) t.eq("gunicorn", c.proc_name)
# Not a config property # Not a config property
t.raises(AttributeError, getattr, c, "foo") t.raises(AttributeError, getattr, c, "foo")
# Force to be not an error # Force to be not an error
@ -93,10 +83,10 @@ def test_property_access():
# Attempt to set a cfg not via c.set # Attempt to set a cfg not via c.set
t.raises(AttributeError, setattr, c, "proc_name", "baz") t.raises(AttributeError, setattr, c, "proc_name", "baz")
# No setting for name # No setting for name
t.raises(AttributeError, c.set, "baz", "bar") t.raises(AttributeError, c.set, "baz", "bar")
def test_bool_validation(): def test_bool_validation():
c = config.Config() c = config.Config()
t.eq(c.debug, False) t.eq(c.debug, False)
@ -196,30 +186,9 @@ def test_load_config():
t.eq(app.cfg.bind, "unix:/tmp/bar/baz") t.eq(app.cfg.bind, "unix:/tmp/bar/baz")
t.eq(app.cfg.workers, 3) t.eq(app.cfg.workers, 3)
t.eq(app.cfg.proc_name, "fooey") t.eq(app.cfg.proc_name, "fooey")
def test_cli_overrides_config(): def test_cli_overrides_config():
with AltArgs(["prog_name", "-c", cfg_file(), "-b", "blarney"]): with AltArgs(["prog_name", "-c", cfg_file(), "-b", "blarney"]):
app = NoConfigApp() app = NoConfigApp()
t.eq(app.cfg.bind, "blarney") t.eq(app.cfg.bind, "blarney")
t.eq(app.cfg.proc_name, "fooey") t.eq(app.cfg.proc_name, "fooey")
def test_paster_config():
with AltArgs(["prog_name", paster_ini()]):
app = PasterApp()
t.eq(app.cfg.bind, "192.168.0.1:80")
t.eq(app.cfg.proc_name, "brim")
t.eq("ignore_me" in app.cfg.settings, False)
def test_cfg_over_paster():
with AltArgs(["prog_name", "-c", cfg_file(), paster_ini()]):
app = PasterApp()
t.eq(app.cfg.bind, "unix:/tmp/bar/baz")
t.eq(app.cfg.proc_name, "fooey")
t.eq(app.cfg.default_proc_name, "blurgh")
def test_cli_cfg_paster():
with AltArgs(["prog_name", "-c", cfg_file(), "-b", "whee", paster_ini()]):
app = PasterApp()
t.eq(app.cfg.bind, "whee")
t.eq(app.cfg.proc_name, "fooey")
t.eq(app.cfg.default_proc_name, "blurgh")

61
tests/test_004-http-body.py Normal file
View File

@ -0,0 +1,61 @@
import t
from gunicorn.http.body import Body
from gunicorn.six import BytesIO
def assert_readline(payload, size, expected):
body = Body(BytesIO(payload))
t.eq(body.readline(size), expected)
def test_readline_empty_body():
assert_readline(b"", None, b"")
assert_readline(b"", 1, b"")
def test_readline_zero_size():
assert_readline(b"abc", 0, b"")
assert_readline(b"\n", 0, b"")
def test_readline_new_line_before_size():
body = Body(BytesIO(b"abc\ndef"))
t.eq(body.readline(4), b"abc\n")
t.eq(body.readline(), b"def")
def test_readline_new_line_after_size():
body = Body(BytesIO(b"abc\ndef"))
t.eq(body.readline(2), b"ab")
t.eq(body.readline(), b"c\n")
def test_readline_no_new_line():
body = Body(BytesIO(b"abcdef"))
t.eq(body.readline(), b"abcdef")
body = Body(BytesIO(b"abcdef"))
t.eq(body.readline(2), b"ab")
t.eq(body.readline(2), b"cd")
t.eq(body.readline(2), b"ef")
def test_readline_buffer_loaded():
reader = BytesIO(b"abc\ndef")
body = Body(reader)
body.read(1) # load internal buffer
reader.write(b"g\nhi")
reader.seek(7)
print(reader.getvalue())
t.eq(body.readline(), b"bc\n")
t.eq(body.readline(), b"defg\n")
t.eq(body.readline(), b"hi")
def test_readline_buffer_loaded_with_size():
body = Body(BytesIO(b"abc\ndef"))
body.read(1) # load internal buffer
t.eq(body.readline(2), b"bc")
t.eq(body.readline(2), b"\n")
t.eq(body.readline(2), b"de")
t.eq(body.readline(2), b"f")

13
tests/test_005-lazywriter-isatty.py Normal file
View File

@ -0,0 +1,13 @@
import sys
from gunicorn.glogging import LazyWriter
def test_lazywriter_isatty():
orig = sys.stdout
sys.stdout = LazyWriter('test.log')
try:
sys.stdout.isatty()
except AttributeError:
raise AssertionError("LazyWriter has no attribute 'isatty'")
sys.stdout = orig

49
tests/treq.py
View File

@ -10,18 +10,19 @@ import t
import inspect import inspect
import os import os
import random import random
import urlparse
from gunicorn.config import Config from gunicorn.config import Config
from gunicorn.http.errors import ParseException from gunicorn.http.errors import ParseException
from gunicorn.http.parser import RequestParser from gunicorn.http.parser import RequestParser
from gunicorn.six import urlparse, execfile_
from gunicorn import six
dirname = os.path.dirname(__file__) dirname = os.path.dirname(__file__)
random.seed() random.seed()
def uri(data): def uri(data):
ret = {"raw": data} ret = {"raw": data}
parts = urlparse.urlparse(data) parts = urlparse(data)
ret["scheme"] = parts.scheme or '' ret["scheme"] = parts.scheme or ''
ret["host"] = parts.netloc.rsplit(":", 1)[0] or None ret["host"] = parts.netloc.rsplit(":", 1)[0] or None
ret["port"] = parts.port or 80 ret["port"] = parts.port or 80
@ -42,7 +43,7 @@ def load_py(fname):
config = globals().copy() config = globals().copy()
config["uri"] = uri config["uri"] = uri
config["cfg"] = Config() config["cfg"] = Config()
execfile(fname, config) execfile_(fname, config)
return config return config
class request(object): class request(object):
@ -54,10 +55,10 @@ class request(object):
if not isinstance(self.expect, list): if not isinstance(self.expect, list):
self.expect = [self.expect] self.expect = [self.expect]
with open(self.fname) as handle: with open(self.fname, 'rb') as handle:
self.data = handle.read() self.data = handle.read()
self.data = self.data.replace("\n", "").replace("\\r\\n", "\r\n") self.data = self.data.replace(b"\n", b"").replace(b"\\r\\n", b"\r\n")
self.data = self.data.replace("\\0", "\000") self.data = self.data.replace(b"\\0", b"\000")
# Functions for sending data to the parser. # Functions for sending data to the parser.
# These functions mock out reading from a # These functions mock out reading from a
@ -69,20 +70,20 @@ class request(object):
def send_lines(self): def send_lines(self):
lines = self.data lines = self.data
pos = lines.find("\r\n") pos = lines.find(b"\r\n")
while pos > 0: while pos > 0:
yield lines[:pos+2] yield lines[:pos+2]
lines = lines[pos+2:] lines = lines[pos+2:]
pos = lines.find("\r\n") pos = lines.find(b"\r\n")
if len(lines): if len(lines):
yield lines yield lines
def send_bytes(self): def send_bytes(self):
for d in self.data: for d in str(self.data.decode("latin1")):
yield d yield bytes(d.encode("latin1"))
def send_random(self): def send_random(self):
maxs = len(self.data) / 10 maxs = round(len(self.data) / 10)
read = 0 read = 0
while read < len(self.data): while read < len(self.data):
chunk = random.randint(1, maxs) chunk = random.randint(1, maxs)
@ -143,7 +144,7 @@ class request(object):
while len(body): while len(body):
if body[:len(data)] != data: if body[:len(data)] != data:
raise AssertionError("Invalid data read: %r" % data) raise AssertionError("Invalid data read: %r" % data)
if '\n' in data[:-1]: if b'\n' in data[:-1]:
raise AssertionError("Embedded new line: %r" % data) raise AssertionError("Embedded new line: %r" % data)
body = body[len(data):] body = body[len(data):]
data = self.szread(req.body.readline, sizes) data = self.szread(req.body.readline, sizes)
@ -165,7 +166,7 @@ class request(object):
""" """
data = req.body.readlines() data = req.body.readlines()
for line in data: for line in data:
if '\n' in line[:-1]: if b'\n' in line[:-1]:
raise AssertionError("Embedded new line: %r" % line) raise AssertionError("Embedded new line: %r" % line)
if line != body[:len(line)]: if line != body[:len(line)]:
raise AssertionError("Invalid body data read: %r != %r" % ( raise AssertionError("Invalid body data read: %r != %r" % (
@ -182,7 +183,7 @@ class request(object):
This skips sizes because there's its not part of the iter api. This skips sizes because there's its not part of the iter api.
""" """
for line in req.body: for line in req.body:
if '\n' in line[:-1]: if b'\n' in line[:-1]:
raise AssertionError("Embedded new line: %r" % line) raise AssertionError("Embedded new line: %r" % line)
if line != body[:len(line)]: if line != body[:len(line)]:
raise AssertionError("Invalid body data read: %r != %r" % ( raise AssertionError("Invalid body data read: %r != %r" % (
@ -191,7 +192,7 @@ class request(object):
if len(body): if len(body):
raise AssertionError("Failed to read entire body: %r" % body) raise AssertionError("Failed to read entire body: %r" % body)
try: try:
data = iter(req.body).next() data = six.next(iter(req.body))
raise AssertionError("Read data after body finished: %r" % data) raise AssertionError("Read data after body finished: %r" % data)
except StopIteration: except StopIteration:
pass pass
@ -214,9 +215,15 @@ class request(object):
ret = [] ret = []
for (mt, sz, sn) in cfgs: for (mt, sz, sn) in cfgs:
mtn = mt.func_name[6:] if hasattr(mt, 'funcname'):
szn = sz.func_name[5:] mtn = mt.func_name[6:]
snn = sn.func_name[5:] szn = sz.func_name[5:]
snn = sn.func_name[5:]
else:
mtn = mt.__name__[6:]
szn = sz.__name__[5:]
snn = sn.__name__[5:]
def test_req(sn, sz, mt): def test_req(sn, sz, mt):
self.check(cfg, sn, sz, mt) self.check(cfg, sn, sz, mt)
desc = "%s: MT: %s SZ: %s SN: %s" % (self.name, mtn, szn, snn) desc = "%s: MT: %s SZ: %s SN: %s" % (self.name, mtn, szn, snn)
@ -251,9 +258,10 @@ class badrequest(object):
self.data = handle.read() self.data = handle.read()
self.data = self.data.replace("\n", "").replace("\\r\\n", "\r\n") self.data = self.data.replace("\n", "").replace("\\r\\n", "\r\n")
self.data = self.data.replace("\\0", "\000") self.data = self.data.replace("\\0", "\000")
self.data = self.data.encode('latin1')
def send(self): def send(self):
maxs = len(self.data) / 10 maxs = round(len(self.data) / 10)
read = 0 read = 0
while read < len(self.data): while read < len(self.data):
chunk = random.randint(1, maxs) chunk = random.randint(1, maxs)
@ -262,5 +270,4 @@ class badrequest(object):
def check(self, cfg): def check(self, cfg):
p = RequestParser(cfg, self.send()) p = RequestParser(cfg, self.send())
[req for req in p] six.next(p)

6
tox.ini
View File

@ -4,8 +4,8 @@
# and then run "tox" from this directory. # and then run "tox" from this directory.
[tox] [tox]
envlist = py26, py27, pypy envlist = py26, py27, py33, pypy
[testenv] [testenv]
commands = nosetests commands = py.test tests/
deps = nose deps = pytest