gronod/headphones
1
0
Fork 1
mirror of https://github.com/rembo10/headphones.git synced 2026-05-15 16:19:28 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/develop' into feature/refactor_config

Browse Source
This commit is contained in:
Jesse Mullan
2014-10-20 00:13:02 -07:00
parent 2686b4c190 cf181e1d26
commit 8fbef76dc0
55 changed files with 4305 additions and 1324 deletions
Download Patch File Download Diff File Expand all files Collapse all files

109
API.md Normal file
View File

@@ -0,0 +1,109 @@
# API Reference
The API is still pretty new and needs some serious cleaning up on the backend but should be reasonably functional. There are no error codes yet.
## General structure
The API endpoint is `http://ip:port + HTTP_ROOT + /api?apikey=$apikey&cmd=$command`
Data response in JSON formatted. If executing a command like "delArtist" or "addArtist" you'll get back an "OK", else, you'll get the data you requested.
## API methods
### getIndex
Fetch data from index page. Returns: ArtistName, ArtistSortName, ArtistID, Status, DateAdded, [LatestAlbum, ReleaseDate, AlbumID], HaveTracks, TotalTracks, IncludeExtras, LastUpdated, [ArtworkURL, ThumbURL]: a remote url to the artwork/thumbnail.
To get the cached image path, see getArtistArt command. ThumbURL is added/updated when an artist is added/updated. If your using the database method to get the artwork, it's more reliable to use the ThumbURL than the ArtworkURL)
### getArtist&id=$artistid
Fetch artist data. returns the artist object (see above) and album info: Status, AlbumASIN, DateAdded, AlbumTitle, ArtistName, ReleaseDate, AlbumID, ArtistID, Type, ArtworkURL: hosted image path. For cached image, see getAlbumArt command)
### getAlbum&id=$albumid
Fetch data from album page. Returns the album object, a description object and a tracks object. Tracks contain: AlbumASIN, AlbumTitle, TrackID, Format, TrackDuration (ms), ArtistName, TrackTitle, AlbumID, ArtistID, Location, TrackNumber, CleanName (stripped of punctuation /styling), BitRate)
### getUpcoming
Returns: Status, AlbumASIN, DateAdded, AlbumTitle, ArtistName, ReleaseDate, AlbumID, ArtistID, Type
### getWanted
Returns: Status, AlbumASIN, DateAdded, AlbumTitle, ArtistName, ReleaseDate, AlbumID, ArtistID, Type
### getSimilar
Returns similar artists - with a higher "Count" being more likely to be similar. Returns: Count, ArtistName, ArtistID
### getHistory
Returns: Status, DateAdded, Title, URL (nzb), FolderName, AlbumID, Size (bytes)
### getLogs
Not working yet
### findArtist&name=$artistname[&limit=$limit]
Perform artist query on musicbrainz. Returns: url, score, name, uniquename (contains disambiguation info), id)
### findAlbum&name=$albumname[&limit=$limit]
Perform album query on musicbrainz. Returns: title, url (artist), id (artist), albumurl, albumid, score, uniquename (artist - with disambiguation)
### addArtist&id=$artistid
Add an artist to the db by artistid)
### addAlbum&id=$releaseid
Add an album to the db by album release id
### delArtist&id=$artistid
Delete artist from db by artistid)
### pauseArtist&id=$artistid
Pause an artist in db)
### resumeArtist&id=$artistid
Resume an artist in db)
### refreshArtist&id=$artistid
Refresh info for artist in db from musicbrainz
### queueAlbum&id=$albumid[&new=True&lossless=True]
Mark an album as wanted and start the searcher. Optional paramters: 'new' looks for new versions, 'lossless' looks only for lossless versions
### unqueueAlbum&id=$albumid
Unmark album as wanted / i.e. mark as skipped
### forceSearch
force search for wanted albums - not launched in a separate thread so it may take a bit to complete
### forceProcess
Force post process albums in download directory - also not launched in a separate thread
### getVersion
Returns some version information: git_path, install_type, current_version, installed_version, commits_behind
### checkGithub
Updates the version information above and returns getVersion data
### shutdown
Shut down headphones
### restart
Restart headphones
### update
Update headphones - you may want to check the install type in get version and not allow this if type==exe
### getArtistArt&id=$artistid
Returns either a relative path to the cached image, or a remote url if the image can't be saved to the cache dir
getAlbumArt&id=$albumid
see above
### getArtistInfo&id=$artistid
Returns Summary and Content, both formatted in html.
### getAlbumInfo&id=$albumid
See above, returns Summary and Content.
### getArtistThumb&id=$artistid
Returns either a relative path to the cached thumbnail artist image, or an http:// address if the cache dir can't be written to.
### getAlbumThumb&id=$albumid
See above.
### choose_specific_download&id=$albumid
Gives you a list of results from searcher.searchforalbum(). Basically runs a normal search, but rather than sorting them and downloading the best result, it dumps the data, which you can then pass on to download_specific_release(). Returns a list of dictionaries with params: title, size, url, provider & kind - all of these values must be passed back to download_specific_release
### download_specific_release&id=albumid&title=$title&size=$size&url=$url&provider=$provider&kind=$kind
Allows you to manually pass a choose_specific_download release back to searcher.send_to_downloader()

71
API_REFERENCE
View File

@@ -1,71 +0,0 @@
The API is still pretty new and needs some serious cleaning up on the backend but should be
reasonably functional. There are no error codes yet,
General structure:
http://localhost:8181 + HTTP_ROOT + /api?apikey=$apikey&cmd=$command
Data returned in json format. If executing a command like "delArtist" or "addArtist" you'll get back an "OK", else, you'll get the data you requested
$commands&parameters[&optionalparameters]:
getIndex (fetch data from index page. Returns: ArtistName, ArtistSortName, ArtistID, Status, DateAdded,
[LatestAlbum, ReleaseDate, AlbumID], HaveTracks, TotalTracks,
IncludeExtras, LastUpdated, [ArtworkURL, ThumbURL]: a remote url to the artwork/thumbnail. To get the cached image path, see getArtistArt command.
ThumbURL is added/updated when an artist is added/updated. If your using the database method to get the artwork,
it's more reliable to use the ThumbURL than the ArtworkURL)
getArtist&id=$artistid (fetch artist data. returns the artist object (see above) and album info: Status, AlbumASIN, DateAdded, AlbumTitle, ArtistName, ReleaseDate, AlbumID, ArtistID, Type, ArtworkURL: hosted image path. For cached image, see getAlbumArt command)
getAlbum&id=$albumid (fetch data from album page. Returns the album object, a description object and a tracks object. Tracks contain: AlbumASIN, AlbumTitle, TrackID, Format, TrackDuration (ms), ArtistName, TrackTitle, AlbumID, ArtistID, Location, TrackNumber, CleanName (stripped of punctuation /styling), BitRate)
getUpcoming (Returns: Status, AlbumASIN, DateAdded, AlbumTitle, ArtistName, ReleaseDate, AlbumID, ArtistID, Type)
getWanted (Returns: Status, AlbumASIN, DateAdded, AlbumTitle, ArtistName, ReleaseDate, AlbumID, ArtistID, Type)
getSimilar (Returns similar artists - with a higher "Count" being more likely to be similar. Returns: Count, ArtistName, ArtistID)
getHistory (Returns: Status, DateAdded, Title, URL (nzb), FolderName, AlbumID, Size (bytes))
getLogs (not working yet)
findArtist&name=$artistname[&limit=$limit] (perform artist query on musicbrainz. Returns: url, score, name, uniquename (contains disambiguation info), id)
findAlbum&name=$albumname[&limit=$limit] (perform album query on musicbrainz. Returns: title, url (artist), id (artist), albumurl, albumid, score, uniquename (artist - with disambiguation)
addArtist&id=$artistid (add an artist to the db by artistid)
addAlbum&id=$releaseid (add an album to the db by album release id)
delArtist&id=$artistid (delete artist from db by artistid)
pauseArtist&id=$artistid (pause an artist in db)
resumeArtist&id=$artistid (resume an artist in db)
refreshArtist&id=$artistid (refresh info for artist in db from musicbrainz)
queueAlbum&id=$albumid[&new=True&lossless=True] (Mark an album as wanted and start the searcher. Optional paramters: 'new' looks for new versions, 'lossless' looks only for lossless versions)
unqueueAlbum&id=$albumid (Unmark album as wanted / i.e. mark as skipped)
forceSearch (force search for wanted albums - not launched in a separate thread so it may take a bit to complete)
forceProcess (force post process albums in download directory - also not launched in a separate thread)
getVersion (Returns some version information: git_path, install_type, current_version, installed_version, commits_behind
checkGithub (updates the version information above and returns getVersion data)
shutdown (shut down headphones)
restart (restart headphones)
update (update headphones - you may want to check the install type in get version and not allow this if type==exe)
getArtistArt&id=$artistid (Returns either a relative path to the cached image, or a remote url if the image can't be saved to the cache dir)
getAlbumArt&id=$albumid (see above)
getArtistInfo&id=$artistid (Returns Summary and Content, both formatted in html)
getAlbumInfo&id=$albumid (See above, returns Summary and Content)
getArtistThumb&id=$artistid (Returns either a relative path to the cached thumbnail artist image, or an http:// address if the cache dir can't be written to)
getAlbumThumb&id=$albumid (see above)
choose_specific_download&id=$albumid (Gives you a list of results from searcher.searchforalbum(). Basically runs a normal search, but rather than sorting them and downloading the best result, it dumps the data, which you can then pass on to download_specific_release(). Returns a list of dictionaries with params: title, size, url, provider & kind - all of these values must be passed back to download_specific_release)
download_specific_release&id=albumid&title=$title&size=$size&url=$url&provider=$provider&kind=$kind (Allows you to manually pass a choose_specific_download release back to searcher.send_to_downloader())

35
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,35 @@
# Contributing to Headphones
## For users
In case you read this because you are posting an issue, please take a minute and conside the things below. The issue tracker is not a support forum. It is primarily intended to submit bugs, improvements or feature requests. However, we are glad to help you, and make sure the problem is not caused by Headphones, but don't expect step-by-step answers.
* Use the search function. Chances are that your problem is already discussed.
* Visit the [Troubleshooting](../../wiki/TroubleShooting) wiki first.
* Use [proper formatting](https://help.github.com/articles/github-flavored-markdown/). Paste your logs in code blocks.
* Close your issue if you resolved it.
## For developers
If you think you can contribute code to the Headphones repository, do not hesitate to submit a pull request.
### Branches
All pull requests should be based on the `develop` branch. When you want to develop a new feature, clone the repository with `git clone origin/develop -b FEATURE_NAME`. Use meaningful commit messages.
### Code compatibility
The code should work with Python 2.6 and 2.7. Note that Headphones runs on different platforms, including Network Attached Storage devices such as Synology.
Re-use existing code. Do not hesitate to add logging in your code. You can the logger module `headphones.logger.*` for this. Web requests are invoked via `headphones.request.*` and derived ones. Use these methods to automatically add proper and meaningful error handling.
### Code conventions
Altough Headphones did not adapt a code convention in the past, we try to follow the [PEP8](http://legacy.python.org/dev/peps/pep-0008/) conventions for future code. A short summary to remind you (copied from http://wiki.ros.org/PyStyleGuide):
* 4 space indentation
* 80 characters per line
* `package_name`
* `ClassName`
* `method_name`
* `field_name`
* `_private_something`
* `self.__really_private_field`
* `_global`
Document your code!

0
COPYING → LICENSE
View File

13
README.md
View File

@@ -1,17 +1,17 @@
#![preview thumb](https://github.com/rembo10/headphones/raw/master/data/images/headphoneslogo.png)Headphones
###Support & Discuss
## Support & Discuss
You are free to join the HP support community on IRC where you can ask questions, hang around and discuss anything related to HP.
1. Use any IRC client and connect to the Freenode server.
2. Join #headphones
###Installation and Notes
## Installation and Notes
[Read our Wiki](../../wiki) on how to install and use HeadPhones properly.
[Read our Wiki](../../wiki) on how to install and configure HeadPhones properly.
[**Troubleshooting** page](../../wiki/TroubleShooting) in the wiki can help you with comon problems.
[**Troubleshooting** page](../../wiki/TroubleShooting) in the wiki can help you with common problems.
**Issues** can be reported on the GitHub issue tracker considering these rules:
@@ -30,8 +30,7 @@ If you **comply with these rules** you can [post your request/issue](http://gith
**Support** the project by implementing new features, solving support tickets and provide bug fixes.
If you change something in the code always make a PR to the developer branch instead of the master branch.
###Screenshots
## Screenshots
Homepage (Artist Overview)
@@ -62,4 +61,4 @@ Album Page with track overview:
![preview thumb](http://i.imgur.com/kcjES.png)
This is free software under the GPL v3 open source license - so feel free to do with it what you wish.
This is free software under the GPL v3 open source license - so feel free to do with it what you wish. A copy of the license is included.

6
headphones/webstart.py
View File

@@ -114,13 +114,13 @@ def initialize(options=None):
# Prevent time-outs
cherrypy.engine.timeout_monitor.unsubscribe()
cherrypy.tree.mount(WebInterface(), options['http_root'], config=conf)
cherrypy.tree.mount(WebInterface(), str(options['http_root']), config=conf)
try:
cherrypy.process.servers.check_port(options['http_host'], options['http_port'])
cherrypy.process.servers.check_port(str(options['http_host']), options['http_port'])
cherrypy.server.start()
except IOError:
sys.stderr.write('Failed to start on port: %i. Is something else running?\n' % (options['http_port']))
sys.exit(1)
cherrypy.server.wait()
cherrypy.server.wait()

76
lib/cherrypy/__init__.py
View File

@@ -53,11 +53,10 @@ with customized or extended components. The core API's are:
* Server API
* WSGI API
These API's are described in the CherryPy specification:
http://www.cherrypy.org/wiki/CherryPySpec
These API's are described in the `CherryPy specification <https://bitbucket.org/cherrypy/cherrypy/wiki/CherryPySpec>`_.
"""
__version__ = "3.2.2"
__version__ = "3.6.0"
from cherrypy._cpcompat import urljoin as _urljoin, urlencode as _urlencode
from cherrypy._cpcompat import basestring, unicodestr, set
@@ -94,6 +93,7 @@ except ImportError:
engine.listeners['before_request'] = set()
engine.listeners['after_request'] = set()
class _TimeoutMonitor(process.plugins.Monitor):
def __init__(self, bus):
@@ -125,6 +125,24 @@ engine.thread_manager.subscribe()
engine.signal_handler = process.plugins.SignalHandler(engine)
class _HandleSignalsPlugin(object):
"""Handle signals from other processes based on the configured
platform handlers above."""
def __init__(self, bus):
self.bus = bus
def subscribe(self):
"""Add the handlers based on the platform"""
if hasattr(self.bus, "signal_handler"):
self.bus.signal_handler.subscribe()
if hasattr(self.bus, "console_control_handler"):
self.bus.console_control_handler.subscribe()
engine.signals = _HandleSignalsPlugin(engine)
from cherrypy import _cpserver
server = _cpserver.Server()
server.subscribe()
@@ -152,18 +170,16 @@ def quickstart(root=None, script_name="", config=None):
tree.mount(root, script_name, config)
if hasattr(engine, "signal_handler"):
engine.signal_handler.subscribe()
if hasattr(engine, "console_control_handler"):
engine.console_control_handler.subscribe()
engine.signals.subscribe()
engine.start()
engine.block()
from cherrypy._cpcompat import threadlocal as _local
class _Serving(_local):
"""An interface for registering request and response objects.
Rather than have a separate "thread local" object for the request and
@@ -258,7 +274,10 @@ request = _ThreadLocalProxy('request')
response = _ThreadLocalProxy('response')
# Create thread_data object as a thread-specific all-purpose storage
class _ThreadData(_local):
"""A container for thread-specific data."""
thread_data = _ThreadData()
@@ -283,7 +302,9 @@ except ImportError:
from cherrypy import _cplogging
class _GlobalLogManager(_cplogging.LogManager):
"""A site-wide LogManager; routes to app.log or global log as appropriate.
This :class:`LogManager<cherrypy._cplogging.LogManager>` implements
@@ -294,8 +315,10 @@ class _GlobalLogManager(_cplogging.LogManager):
"""
def __call__(self, *args, **kwargs):
"""Log the given message to the app.log or global log as appropriate."""
# Do NOT use try/except here. See http://www.cherrypy.org/ticket/945
"""Log the given message to the app.log or global log as appropriate.
"""
# Do NOT use try/except here. See
# https://bitbucket.org/cherrypy/cherrypy/issue/945
if hasattr(request, 'app') and hasattr(request.app, 'log'):
log = request.app.log
else:
@@ -303,7 +326,8 @@ class _GlobalLogManager(_cplogging.LogManager):
return log.error(*args, **kwargs)
def access(self):
"""Log an access message to the app.log or global log as appropriate."""
"""Log an access message to the app.log or global log as appropriate.
"""
try:
return request.app.log.access()
except AttributeError:
@@ -317,6 +341,7 @@ log.error_file = ''
# Using an access file makes CP about 10% slower. Leave off by default.
log.access_file = ''
def _buslog(msg, level):
log.error(msg, 'ENGINE', severity=level)
engine.subscribe('log', _buslog)
@@ -336,7 +361,8 @@ def expose(func=None, alias=None):
parents[a.replace(".", "_")] = func
return func
import sys, types
import sys
import types
if isinstance(func, (types.FunctionType, types.MethodType)):
if alias is None:
# @expose
@@ -363,6 +389,7 @@ def expose(func=None, alias=None):
alias = func
return expose_
def popargs(*args, **kwargs):
"""A decorator for _cp_dispatch
(cherrypy.dispatch.Dispatcher.dispatch_method_name).
@@ -442,34 +469,34 @@ def popargs(*args, **kwargs):
"""
#Since keyword arg comes after *args, we have to process it ourselves
#for lower versions of python.
# Since keyword arg comes after *args, we have to process it ourselves
# for lower versions of python.
handler = None
handler_call = False
for k,v in kwargs.items():
for k, v in kwargs.items():
if k == 'handler':
handler = v
else:
raise TypeError(
"cherrypy.popargs() got an unexpected keyword argument '{0}'" \
"cherrypy.popargs() got an unexpected keyword argument '{0}'"
.format(k)
)
)
import inspect
if handler is not None \
and (hasattr(handler, '__call__') or inspect.isclass(handler)):
and (hasattr(handler, '__call__') or inspect.isclass(handler)):
handler_call = True
def decorated(cls_or_self=None, vpath=None):
if inspect.isclass(cls_or_self):
#cherrypy.popargs is a class decorator
# cherrypy.popargs is a class decorator
cls = cls_or_self
setattr(cls, dispatch.Dispatcher.dispatch_method_name, decorated)
return cls
#We're in the actual function
# We're in the actual function
self = cls_or_self
parms = {}
for arg in args:
@@ -486,9 +513,9 @@ def popargs(*args, **kwargs):
request.params.update(parms)
#If we are the ultimate handler, then to prevent our _cp_dispatch
#from being called again, we will resolve remaining elements through
#getattr() directly.
# If we are the ultimate handler, then to prevent our _cp_dispatch
# from being called again, we will resolve remaining elements through
# getattr() directly.
if vpath:
return getattr(self, vpath.pop(0), None)
else:
@@ -496,6 +523,7 @@ def popargs(*args, **kwargs):
return decorated
def url(path="", qs="", script_name=None, base=None, relative=None):
"""Create an absolute URL for the given path.
@@ -613,7 +641,7 @@ config.defaults = {
'tools.log_headers.on': True,
'tools.trailing_slash.on': True,
'tools.encode.on': True
}
}
config.namespaces["log"] = lambda k, v: setattr(log, k, v)
config.namespaces["checker"] = lambda k, v: setattr(checker, k, v)
# Must reset to get our defaults applied.

85
lib/cherrypy/_cpchecker.py
View File

@@ -6,6 +6,7 @@ from cherrypy._cpcompat import iteritems, copykeys, builtins
class Checker(object):
"""A checker for CherryPy sites and their mounted applications.
When this object is called at engine startup, it executes each
@@ -22,7 +23,6 @@ class Checker(object):
on = True
"""If True (the default), run all checks; if False, turn off all checks."""
def __init__(self):
self._populate_known_types()
@@ -48,7 +48,8 @@ class Checker(object):
global_config_contained_paths = False
def check_app_config_entries_dont_start_with_script_name(self):
"""Check for Application config with sections that repeat script_name."""
"""Check for Application config with sections that repeat script_name.
"""
for sn, app in cherrypy.tree.apps.items():
if not isinstance(app, cherrypy.Application):
continue
@@ -61,8 +62,9 @@ class Checker(object):
key_atoms = key.strip("/").split("/")
if key_atoms[:len(sn_atoms)] == sn_atoms:
warnings.warn(
"The application mounted at %r has config " \
"entries that start with its script name: %r" % (sn, key))
"The application mounted at %r has config "
"entries that start with its script name: %r" % (sn,
key))
def check_site_config_entries_in_app_config(self):
"""Check for mounted Applications that have site-scoped config."""
@@ -76,13 +78,15 @@ class Checker(object):
for key, value in iteritems(entries):
for n in ("engine.", "server.", "tree.", "checker."):
if key.startswith(n):
msg.append("[%s] %s = %s" % (section, key, value))
msg.append("[%s] %s = %s" %
(section, key, value))
if msg:
msg.insert(0,
"The application mounted at %r contains the following "
"config entries, which are only allowed in site-wide "
"config. Move them to a [global] section and pass them "
"to cherrypy.config.update() instead of tree.mount()." % sn)
"The application mounted at %r contains the "
"following config entries, which are only allowed "
"in site-wide config. Move them to a [global] "
"section and pass them to cherrypy.config.update() "
"instead of tree.mount()." % sn)
warnings.warn(os.linesep.join(msg))
def check_skipped_app_config(self):
@@ -102,7 +106,9 @@ class Checker(object):
return
def check_app_config_brackets(self):
"""Check for Application config with extraneous brackets in section names."""
"""Check for Application config with extraneous brackets in section
names.
"""
for sn, app in cherrypy.tree.apps.items():
if not isinstance(app, cherrypy.Application):
continue
@@ -111,7 +117,7 @@ class Checker(object):
for key in app.config.keys():
if key.startswith("[") or key.endswith("]"):
warnings.warn(
"The application mounted at %r has config " \
"The application mounted at %r has config "
"section names with extraneous brackets: %r. "
"Config *files* need brackets; config *dicts* "
"(e.g. passed to tree.mount) do not." % (sn, key))
@@ -144,16 +150,20 @@ class Checker(object):
"though a root is provided.")
testdir = os.path.join(root, dir[1:])
if os.path.exists(testdir):
msg += ("\nIf you meant to serve the "
"filesystem folder at %r, remove "
"the leading slash from dir." % testdir)
msg += (
"\nIf you meant to serve the "
"filesystem folder at %r, remove the "
"leading slash from dir." % (testdir,))
else:
if not root:
msg = "dir is a relative path and no root provided."
msg = (
"dir is a relative path and "
"no root provided.")
else:
fulldir = os.path.join(root, dir)
if not os.path.isabs(fulldir):
msg = "%r is not an absolute path." % fulldir
msg = ("%r is not an absolute path." % (
fulldir,))
if fulldir and not os.path.exists(fulldir):
if msg:
@@ -165,9 +175,7 @@ class Checker(object):
warnings.warn("%s\nsection: [%s]\nroot: %r\ndir: %r"
% (msg, section, root, dir))
# -------------------------- Compatibility -------------------------- #
obsolete = {
'server.default_content_type': 'tools.response_headers.headers',
'log_access_file': 'log.access_file',
@@ -180,7 +188,7 @@ class Checker(object):
'throw_errors': 'request.throw_errors',
'profiler.on': ('cherrypy.tree.mount(profiler.make_app('
'cherrypy.Application(Root())))'),
}
}
deprecated = {}
@@ -213,9 +221,7 @@ class Checker(object):
continue
self._compat(app.config)
# ------------------------ Known Namespaces ------------------------ #
extra_config_namespaces = []
def _known_ns(self, app):
@@ -235,20 +241,24 @@ class Checker(object):
if atoms[0] not in ns:
# Spit out a special warning if a known
# namespace is preceded by "cherrypy."
if (atoms[0] == "cherrypy" and atoms[1] in ns):
msg = ("The config entry %r is invalid; "
"try %r instead.\nsection: [%s]"
% (k, ".".join(atoms[1:]), section))
if atoms[0] == "cherrypy" and atoms[1] in ns:
msg = (
"The config entry %r is invalid; "
"try %r instead.\nsection: [%s]"
% (k, ".".join(atoms[1:]), section))
else:
msg = ("The config entry %r is invalid, because "
"the %r config namespace is unknown.\n"
"section: [%s]" % (k, atoms[0], section))
msg = (
"The config entry %r is invalid, "
"because the %r config namespace "
"is unknown.\n"
"section: [%s]" % (k, atoms[0], section))
warnings.warn(msg)
elif atoms[0] == "tools":
if atoms[1] not in dir(cherrypy.tools):
msg = ("The config entry %r may be invalid, "
"because the %r tool was not found.\n"
"section: [%s]" % (k, atoms[1], section))
msg = (
"The config entry %r may be invalid, "
"because the %r tool was not found.\n"
"section: [%s]" % (k, atoms[1], section))
warnings.warn(msg)
def check_config_namespaces(self):
@@ -258,11 +268,7 @@ class Checker(object):
continue
self._known_ns(app)
# -------------------------- Config Types -------------------------- #
known_config_types = {}
def _populate_known_types(self):
@@ -314,14 +320,13 @@ class Checker(object):
continue
self._known_types(app.config)
# -------------------- Specific config warnings -------------------- #
def check_localhost(self):
"""Warn if any socket_host is 'localhost'. See #711."""
for k, v in cherrypy.config.items():
if k == 'server.socket_host' and v == 'localhost':
warnings.warn("The use of 'localhost' as a socket host can "
"cause problems on newer systems, since 'localhost' can "
"map to either an IPv4 or an IPv6 address. You should "
"use '127.0.0.1' or '[::1]' instead.")
"cause problems on newer systems, since "
"'localhost' can map to either an IPv4 or an "
"IPv6 address. You should use '127.0.0.1' "
"or '[::1]' instead.")

135
lib/cherrypy/_cpcompat.py
View File

@@ -18,6 +18,7 @@ output.
import os
import re
import sys
import threading
if sys.version_info >= (3, 0):
py3k = True
@@ -25,14 +26,23 @@ if sys.version_info >= (3, 0):
unicodestr = str
nativestr = unicodestr
basestring = (bytes, str)
def ntob(n, encoding='ISO-8859-1'):
"""Return the given native string as a byte string in the given encoding."""
"""Return the given native string as a byte string in the given
encoding.
"""
assert_native(n)
# In Python 3, the native string type is unicode
return n.encode(encoding)
def ntou(n, encoding='ISO-8859-1'):
"""Return the given native string as a unicode string with the given encoding."""
"""Return the given native string as a unicode string with the given
encoding.
"""
assert_native(n)
# In Python 3, the native string type is unicode
return n
def tonative(n, encoding='ISO-8859-1'):
"""Return the given string as a native string in the given encoding."""
# In Python 3, the native string type is unicode
@@ -50,27 +60,36 @@ else:
unicodestr = unicode
nativestr = bytestr
basestring = basestring
def ntob(n, encoding='ISO-8859-1'):
"""Return the given native string as a byte string in the given encoding."""
"""Return the given native string as a byte string in the given
encoding.
"""
assert_native(n)
# In Python 2, the native string type is bytes. Assume it's already
# in the given encoding, which for ISO-8859-1 is almost always what
# was intended.
return n
def ntou(n, encoding='ISO-8859-1'):
"""Return the given native string as a unicode string with the given encoding."""
"""Return the given native string as a unicode string with the given
encoding.
"""
assert_native(n)
# In Python 2, the native string type is bytes.
# First, check for the special encoding 'escape'. The test suite uses this
# to signal that it wants to pass a string with embedded \uXXXX escapes,
# but without having to prefix it with u'' for Python 2, but no prefix
# for Python 3.
# First, check for the special encoding 'escape'. The test suite uses
# this to signal that it wants to pass a string with embedded \uXXXX
# escapes, but without having to prefix it with u'' for Python 2,
# but no prefix for Python 3.
if encoding == 'escape':
return unicode(
re.sub(r'\\u([0-9a-zA-Z]{4})',
lambda m: unichr(int(m.group(1), 16)),
n.decode('ISO-8859-1')))
# Assume it's already in the given encoding, which for ISO-8859-1 is almost
# always what was intended.
# Assume it's already in the given encoding, which for ISO-8859-1
# is almost always what was intended.
return n.decode(encoding)
def tonative(n, encoding='ISO-8859-1'):
"""Return the given string as a native string in the given encoding."""
# In Python 2, the native string type is bytes.
@@ -86,6 +105,11 @@ else:
# bytes:
BytesIO = StringIO
def assert_native(n):
if not isinstance(n, nativestr):
raise TypeError("n must be a native str (got %s)" % type(n).__name__)
try:
set = set
except NameError:
@@ -100,6 +124,7 @@ except ImportError:
# the legacy API of base64
from base64 import decodestring as _base64_decodebytes
def base64_decode(n, encoding='ISO-8859-1'):
"""Return the native string base64-decoded (as a native string)."""
if isinstance(n, unicodestr):
@@ -198,28 +223,31 @@ except ImportError:
import __builtin__ as builtins
try:
# Python 2. We have to do it in this order so Python 2 builds
# Python 2. We try Python 2 first clients on Python 2
# don't try to import the 'http' module from cherrypy.lib
from Cookie import SimpleCookie, CookieError
from httplib import BadStatusLine, HTTPConnection, HTTPSConnection, IncompleteRead, NotConnected
from httplib import BadStatusLine, HTTPConnection, IncompleteRead
from httplib import NotConnected
from BaseHTTPServer import BaseHTTPRequestHandler
except ImportError:
# Python 3
from http.cookies import SimpleCookie, CookieError
from http.client import BadStatusLine, HTTPConnection, HTTPSConnection, IncompleteRead, NotConnected
from http.client import BadStatusLine, HTTPConnection, IncompleteRead
from http.client import NotConnected
from http.server import BaseHTTPRequestHandler
try:
# Python 2. We have to do it in this order so Python 2 builds
# don't try to import the 'http' module from cherrypy.lib
from httplib import HTTPSConnection
except ImportError:
# Some platforms don't expose HTTPSConnection, so handle it separately
if py3k:
try:
# Python 3
from http.client import HTTPSConnection
except ImportError:
# Some platforms which don't have SSL don't expose HTTPSConnection
HTTPSConnection = None
else:
try:
from httplib import HTTPSConnection
except ImportError:
HTTPSConnection = None
try:
# Python 2
@@ -233,16 +261,19 @@ if hasattr(threading.Thread, "daemon"):
# Python 2.6+
def get_daemon(t):
return t.daemon
def set_daemon(t, val):
t.daemon = val
else:
def get_daemon(t):
return t.isDaemon()
def set_daemon(t, val):
t.setDaemon(val)
try:
from email.utils import formatdate
def HTTPDate(timeval=None):
return formatdate(timeval, usegmt=True)
except ImportError:
@@ -251,40 +282,49 @@ except ImportError:
try:
# Python 3
from urllib.parse import unquote as parse_unquote
def unquote_qs(atom, encoding, errors='strict'):
return parse_unquote(atom.replace('+', ' '), encoding=encoding, errors=errors)
return parse_unquote(
atom.replace('+', ' '),
encoding=encoding,
errors=errors)
except ImportError:
# Python 2
from urllib import unquote as parse_unquote
def unquote_qs(atom, encoding, errors='strict'):
return parse_unquote(atom.replace('+', ' ')).decode(encoding, errors)
try:
# Prefer simplejson, which is usually more advanced than the builtin module.
# Prefer simplejson, which is usually more advanced than the builtin
# module.
import simplejson as json
json_decode = json.JSONDecoder().decode
json_encode = json.JSONEncoder().iterencode
_json_encode = json.JSONEncoder().iterencode
except ImportError:
if py3k:
# Python 3.0: json is part of the standard library,
# but outputs unicode. We need bytes.
if sys.version_info >= (2, 6):
# Python >=2.6 : json is part of the standard library
import json
json_decode = json.JSONDecoder().decode
_json_encode = json.JSONEncoder().iterencode
else:
json = None
def json_decode(s):
raise ValueError('No JSON library is available')
def _json_encode(s):
raise ValueError('No JSON library is available')
finally:
if json and py3k:
# The two Python 3 implementations (simplejson/json)
# outputs str. We need bytes.
def json_encode(value):
for chunk in _json_encode(value):
yield chunk.encode('utf8')
elif sys.version_info >= (2, 6):
# Python 2.6: json is part of the standard library
import json
json_decode = json.JSONDecoder().decode
json_encode = json.JSONEncoder().iterencode
else:
json = None
def json_decode(s):
raise ValueError('No JSON library is available')
def json_encode(s):
raise ValueError('No JSON library is available')
json_encode = _json_encode
try:
import cPickle as pickle
@@ -296,11 +336,13 @@ except ImportError:
try:
os.urandom(20)
import binascii
def random20():
return binascii.hexlify(os.urandom(20)).decode('ascii')
except (AttributeError, NotImplementedError):
import random
# os.urandom not available until Python 2.4. Fall back to random.random.
def random20():
return sha('%s' % random.random()).hexdigest()
@@ -316,3 +358,26 @@ except NameError:
# Python 2
def next(i):
return i.next()
if sys.version_info >= (3, 3):
Timer = threading.Timer
Event = threading.Event
else:
# Python 3.2 and earlier
Timer = threading._Timer
Event = threading._Event
# Prior to Python 2.6, the Thread class did not have a .daemon property.
# This mix-in adds that property.
class SetDaemonProperty:
def __get_daemon(self):
return self.isDaemon()
def __set_daemon(self, daemon):
self.setDaemon(daemon)
if sys.version_info < (2, 6):
daemon = property(__get_daemon, __set_daemon)

1544
lib/cherrypy/_cpcompat_subprocess.py Normal file
View File

File diff suppressed because it is too large Load Diff

50
lib/cherrypy/_cpconfig.py
View File

@@ -125,6 +125,7 @@ from cherrypy.lib import reprconf
# Deprecated in CherryPy 3.2--remove in 3.3
NamespaceSet = reprconf.NamespaceSet
def merge(base, other):
"""Merge one app config (from a dict, file, or filename) into another.
@@ -146,6 +147,7 @@ def merge(base, other):
class Config(reprconf.Config):
"""The 'global' configuration data for the entire CherryPy process."""
def update(self, config):
@@ -157,7 +159,7 @@ class Config(reprconf.Config):
def _apply(self, config):
"""Update self from a dict."""
if isinstance(config.get("global", None), dict):
if isinstance(config.get("global"), dict):
if len(config) > 1:
cherrypy.checker.global_config_contained_paths = True
config = config["global"]
@@ -171,6 +173,7 @@ class Config(reprconf.Config):
raise TypeError(
"The cherrypy.config decorator does not accept positional "
"arguments; you must use keyword arguments.")
def tool_decorator(f):
if not hasattr(f, "_cp_config"):
f._cp_config = {}
@@ -180,25 +183,26 @@ class Config(reprconf.Config):
return tool_decorator
# Sphinx begin config.environments
Config.environments = environments = {
"staging": {
'engine.autoreload_on': False,
'engine.autoreload.on': False,
'checker.on': False,
'tools.log_headers.on': False,
'request.show_tracebacks': False,
'request.show_mismatched_params': False,
},
},
"production": {
'engine.autoreload_on': False,
'engine.autoreload.on': False,
'checker.on': False,
'tools.log_headers.on': False,
'request.show_tracebacks': False,
'request.show_mismatched_params': False,
'log.screen': False,
},
},
"embedded": {
# For use with CherryPy embedded in another deployment stack.
'engine.autoreload_on': False,
'engine.autoreload.on': False,
'checker.on': False,
'tools.log_headers.on': False,
'request.show_tracebacks': False,
@@ -206,16 +210,17 @@ Config.environments = environments = {
'log.screen': False,
'engine.SIGHUP': None,
'engine.SIGTERM': None,
},
},
"test_suite": {
'engine.autoreload_on': False,
'engine.autoreload.on': False,
'checker.on': False,
'tools.log_headers.on': False,
'request.show_tracebacks': True,
'request.show_mismatched_params': True,
'log.screen': False,
},
}
},
}
# Sphinx end config.environments
def _server_namespace_handler(k, v):
@@ -245,9 +250,24 @@ def _server_namespace_handler(k, v):
setattr(cherrypy.server, k, v)
Config.namespaces["server"] = _server_namespace_handler
def _engine_namespace_handler(k, v):
"""Backward compatibility handler for the "engine" namespace."""
engine = cherrypy.engine
deprecated = {
'autoreload_on': 'autoreload.on',
'autoreload_frequency': 'autoreload.frequency',
'autoreload_match': 'autoreload.match',
'reload_files': 'autoreload.files',
'deadlock_poll_freq': 'timeout_monitor.frequency'
}
if k in deprecated:
engine.log(
'WARNING: Use of engine.%s is deprecated and will be removed in a '
'future version. Use engine.%s instead.' % (k, deprecated[k]))
if k == 'autoreload_on':
if v:
engine.autoreload.subscribe()
@@ -272,7 +292,10 @@ def _engine_namespace_handler(k, v):
if v and hasattr(getattr(plugin, 'subscribe', None), '__call__'):
plugin.subscribe()
return
elif (not v) and hasattr(getattr(plugin, 'unsubscribe', None), '__call__'):
elif (
(not v) and
hasattr(getattr(plugin, 'unsubscribe', None), '__call__')
):
plugin.unsubscribe()
return
setattr(plugin, attrname, v)
@@ -286,10 +309,9 @@ def _tree_namespace_handler(k, v):
if isinstance(v, dict):
for script_name, app in v.items():
cherrypy.tree.graft(app, script_name)
cherrypy.engine.log("Mounted: %s on %s" % (app, script_name or "/"))
cherrypy.engine.log("Mounted: %s on %s" %
(app, script_name or "/"))
else:
cherrypy.tree.graft(v, v.script_name)
cherrypy.engine.log("Mounted: %s on %s" % (v, v.script_name or "/"))
Config.namespaces["tree"] = _tree_namespace_handler

116
lib/cherrypy/_cpdispatch.py
View File

@@ -22,6 +22,7 @@ from cherrypy._cpcompat import set
class PageHandler(object):
"""Callable which sets response.body."""
def __init__(self, callable, *args, **kwargs):
@@ -29,6 +30,32 @@ class PageHandler(object):
self.args = args
self.kwargs = kwargs
def get_args(self):
return cherrypy.serving.request.args
def set_args(self, args):
cherrypy.serving.request.args = args
return cherrypy.serving.request.args
args = property(
get_args,
set_args,
doc="The ordered args should be accessible from post dispatch hooks"
)
def get_kwargs(self):
return cherrypy.serving.request.kwargs
def set_kwargs(self, kwargs):
cherrypy.serving.request.kwargs = kwargs
return cherrypy.serving.request.kwargs
kwargs = property(
get_kwargs,
set_kwargs,
doc="The named kwargs should be accessible from post dispatch hooks"
)
def __call__(self):
try:
return self.callable(*self.args, **self.kwargs)
@@ -54,7 +81,8 @@ def test_callable_spec(callable, callable_args, callable_kwargs):
2. Too little parameters are passed to the function.
There are 3 sources of parameters to a cherrypy handler.
1. query string parameters are passed as keyword parameters to the handler.
1. query string parameters are passed as keyword parameters to the
handler.
2. body parameters are also passed as keyword parameters.
3. when partial matching occurs, the final path atoms are passed as
positional args.
@@ -65,10 +93,11 @@ def test_callable_spec(callable, callable_args, callable_kwargs):
show_mismatched_params = getattr(
cherrypy.serving.request, 'show_mismatched_params', False)
try:
(args, varargs, varkw, defaults) = inspect.getargspec(callable)
(args, varargs, varkw, defaults) = getargspec(callable)
except TypeError:
if isinstance(callable, object) and hasattr(callable, '__call__'):
(args, varargs, varkw, defaults) = inspect.getargspec(callable.__call__)
(args, varargs, varkw,
defaults) = getargspec(callable.__call__)
else:
# If it wasn't one of our own types, re-raise
# the original error
@@ -125,7 +154,7 @@ def test_callable_spec(callable, callable_args, callable_kwargs):
# arguments it's definitely a 404.
message = None
if show_mismatched_params:
message="Missing parameters: %s" % ",".join(missing_args)
message = "Missing parameters: %s" % ",".join(missing_args)
raise cherrypy.HTTPError(404, message=message)
# the extra positional arguments come from the path - 404 Not Found
@@ -147,8 +176,8 @@ def test_callable_spec(callable, callable_args, callable_kwargs):
message = None
if show_mismatched_params:
message="Multiple values for parameters: "\
"%s" % ",".join(multiple_args)
message = "Multiple values for parameters: "\
"%s" % ",".join(multiple_args)
raise cherrypy.HTTPError(error, message=message)
if not varkw and varkw_usage > 0:
@@ -158,8 +187,8 @@ def test_callable_spec(callable, callable_args, callable_kwargs):
if extra_qs_params:
message = None
if show_mismatched_params:
message="Unexpected query string "\
"parameters: %s" % ", ".join(extra_qs_params)
message = "Unexpected query string "\
"parameters: %s" % ", ".join(extra_qs_params)
raise cherrypy.HTTPError(404, message=message)
# If there were any extra body parameters, it's a 400 Not Found
@@ -167,8 +196,8 @@ def test_callable_spec(callable, callable_args, callable_kwargs):
if extra_body_params:
message = None
if show_mismatched_params:
message="Unexpected body parameters: "\
"%s" % ", ".join(extra_body_params)
message = "Unexpected body parameters: "\
"%s" % ", ".join(extra_body_params)
raise cherrypy.HTTPError(400, message=message)
@@ -176,10 +205,16 @@ try:
import inspect
except ImportError:
test_callable_spec = lambda callable, args, kwargs: None
else:
getargspec = inspect.getargspec
# Python 3 requires using getfullargspec if keyword-only arguments are present
if hasattr(inspect, 'getfullargspec'):
def getargspec(callable):
return inspect.getfullargspec(callable)[:4]
class LateParamPageHandler(PageHandler):
"""When passing cherrypy.request.params to the page handler, we do not
want to capture that dict too early; we want to give tools like the
decoding tool a chance to modify the params dict in-between the lookup
@@ -195,6 +230,7 @@ class LateParamPageHandler(PageHandler):
return kwargs
def _set_kwargs(self, kwargs):
cherrypy.serving.request.kwargs = kwargs
self._kwargs = kwargs
kwargs = property(_get_kwargs, _set_kwargs,
@@ -205,17 +241,22 @@ class LateParamPageHandler(PageHandler):
if sys.version_info < (3, 0):
punctuation_to_underscores = string.maketrans(
string.punctuation, '_' * len(string.punctuation))
def validate_translator(t):
if not isinstance(t, str) or len(t) != 256:
raise ValueError("The translate argument must be a str of len 256.")
raise ValueError(
"The translate argument must be a str of len 256.")
else:
punctuation_to_underscores = str.maketrans(
string.punctuation, '_' * len(string.punctuation))
def validate_translator(t):
if not isinstance(t, dict):
raise ValueError("The translate argument must be a dict.")
class Dispatcher(object):
"""CherryPy Dispatcher which walks a tree of objects to find a handler.
The tree is rooted at cherrypy.request.app.root, and each hierarchical
@@ -304,31 +345,31 @@ class Dispatcher(object):
if dispatch and hasattr(dispatch, '__call__') and not \
getattr(dispatch, 'exposed', False) and \
pre_len > 1:
#Don't expose the hidden 'index' token to _cp_dispatch
#We skip this if pre_len == 1 since it makes no sense
#to call a dispatcher when we have no tokens left.
# Don't expose the hidden 'index' token to _cp_dispatch
# We skip this if pre_len == 1 since it makes no sense
# to call a dispatcher when we have no tokens left.
index_name = iternames.pop()
subnode = dispatch(vpath=iternames)
iternames.append(index_name)
else:
#We didn't find a path, but keep processing in case there
#is a default() handler.
# We didn't find a path, but keep processing in case there
# is a default() handler.
iternames.pop(0)
else:
#We found the path, remove the vpath entry
# We found the path, remove the vpath entry
iternames.pop(0)
segleft = len(iternames)
if segleft > pre_len:
#No path segment was removed. Raise an error.
# No path segment was removed. Raise an error.
raise cherrypy.CherryPyException(
"A vpath segment was added. Custom dispatchers may only "
+ "remove elements. While trying to process "
+ "{0} in {1}".format(name, fullpath)
)
)
elif segleft == pre_len:
#Assume that the handler used the current path segment, but
#did not pop it. This allows things like
#return getattr(self, vpath[0], None)
# Assume that the handler used the current path segment, but
# did not pop it. This allows things like
# return getattr(self, vpath[0], None)
iternames.pop(0)
segleft -= 1
node = subnode
@@ -353,14 +394,16 @@ class Dispatcher(object):
object_trail.append([name, node, nodeconf, segleft])
def set_conf():
"""Collapse all object_trail config into cherrypy.request.config."""
"""Collapse all object_trail config into cherrypy.request.config.
"""
base = cherrypy.config.copy()
# Note that we merge the config from each node
# even if that node was None.
for name, obj, conf, segleft in object_trail:
base.update(conf)
if 'tools.staticdir.dir' in conf:
base['tools.staticdir.section'] = '/' + '/'.join(fullpath[0:fullpath_len - segleft])
base['tools.staticdir.section'] = '/' + \
'/'.join(fullpath[0:fullpath_len - segleft])
return base
# Try successive objects (reverse order)
@@ -377,13 +420,15 @@ class Dispatcher(object):
if getattr(defhandler, 'exposed', False):
# Insert any extra _cp_config from the default handler.
conf = getattr(defhandler, "_cp_config", {})
object_trail.insert(i+1, ["default", defhandler, conf, segleft])
object_trail.insert(
i + 1, ["default", defhandler, conf, segleft])
request.config = set_conf()
# See http://www.cherrypy.org/ticket/613
# See https://bitbucket.org/cherrypy/cherrypy/issue/613
request.is_index = path.endswith("/")
return defhandler, fullpath[fullpath_len - segleft:-1]
# Uncomment the next line to restrict positional params to "default".
# Uncomment the next line to restrict positional params to
# "default".
# if i < num_candidates - 2: continue
# Try the current leaf.
@@ -407,6 +452,7 @@ class Dispatcher(object):
class MethodDispatcher(Dispatcher):
"""Additional dispatch based on cherrypy.request.method.upper().
Methods named GET, POST, etc will be called on an exposed class.
@@ -450,9 +496,10 @@ class MethodDispatcher(Dispatcher):
class RoutesDispatcher(object):
"""A Routes based dispatcher for CherryPy."""
def __init__(self, full_result=False):
def __init__(self, full_result=False, **mapper_options):
"""
Routes dispatcher
@@ -463,7 +510,7 @@ class RoutesDispatcher(object):
import routes
self.full_result = full_result
self.controllers = {}
self.mapper = routes.Mapper()
self.mapper = routes.Mapper(**mapper_options)
self.mapper.controller_scan = self.controllers.keys
def connect(self, name, route, controller, **kwargs):
@@ -565,13 +612,15 @@ class RoutesDispatcher(object):
def XMLRPCDispatcher(next_dispatcher=Dispatcher()):
from cherrypy.lib import xmlrpcutil
def xmlrpc_dispatch(path_info):
path_info = xmlrpcutil.patched_path(path_info)
return next_dispatcher(path_info)
return xmlrpc_dispatch
def VirtualHost(next_dispatcher=Dispatcher(), use_x_forwarded_host=True, **domains):
def VirtualHost(next_dispatcher=Dispatcher(), use_x_forwarded_host=True,
**domains):
"""
Select a different handler based on the Host header.
@@ -611,6 +660,7 @@ def VirtualHost(next_dispatcher=Dispatcher(), use_x_forwarded_host=True, **domai
headers may contain the port number.
"""
from cherrypy.lib import httputil
def vhost_dispatch(path_info):
request = cherrypy.serving.request
header = request.headers.get
@@ -625,7 +675,8 @@ def VirtualHost(next_dispatcher=Dispatcher(), use_x_forwarded_host=True, **domai
result = next_dispatcher(path_info)
# Touch up staticdir config. See http://www.cherrypy.org/ticket/614.
# Touch up staticdir config. See
# https://bitbucket.org/cherrypy/cherrypy/issue/614.
section = request.config.get('tools.staticdir.section')
if section:
section = section[len(prefix):]
@@ -633,4 +684,3 @@ def VirtualHost(next_dispatcher=Dispatcher(), use_x_forwarded_host=True, **domai
return result
return vhost_dispatch

139
lib/cherrypy/_cperror.py
View File

@@ -2,8 +2,9 @@
CherryPy provides (and uses) exceptions for declaring that the HTTP response
should be a status other than the default "200 OK". You can ``raise`` them like
normal Python exceptions. You can also call them and they will raise themselves;
this means you can set an :class:`HTTPError<cherrypy._cperror.HTTPError>`
normal Python exceptions. You can also call them and they will raise
themselves; this means you can set an
:class:`HTTPError<cherrypy._cperror.HTTPError>`
or :class:`HTTPRedirect<cherrypy._cperror.HTTPRedirect>` as the
:attr:`request.handler<cherrypy._cprequest.Request.handler>`.
@@ -21,7 +22,8 @@ POST, however, is neither safe nor idempotent--if you
charge a credit card, you don't want to be charged twice by a redirect!
For this reason, *none* of the 3xx responses permit a user-agent (browser) to
resubmit a POST on redirection without first confirming the action with the user:
resubmit a POST on redirection without first confirming the action with the
user:
===== ================================= ===========
300 Multiple Choices Confirm with the user
@@ -53,14 +55,16 @@ Anticipated HTTP responses
--------------------------
The 'error_page' config namespace can be used to provide custom HTML output for
expected responses (like 404 Not Found). Supply a filename from which the output
will be read. The contents will be interpolated with the values %(status)s,
%(message)s, %(traceback)s, and %(version)s using plain old Python
`string formatting <http://www.python.org/doc/2.6.4/library/stdtypes.html#string-formatting-operations>`_.
expected responses (like 404 Not Found). Supply a filename from which the
output will be read. The contents will be interpolated with the values
%(status)s, %(message)s, %(traceback)s, and %(version)s using plain old Python
`string formatting <http://docs.python.org/2/library/stdtypes.html#string-formatting-operations>`_.
::
_cp_config = {'error_page.404': os.path.join(localDir, "static/index.html")}
_cp_config = {
'error_page.404': os.path.join(localDir, "static/index.html")
}
Beginning in version 3.1, you may also provide a function or other callable as
@@ -72,7 +76,8 @@ version arguments that are interpolated into templates::
cherrypy.config.update({'error_page.402': error_page_402})
Also in 3.1, in addition to the numbered error codes, you may also supply
"error_page.default" to handle all codes which do not have their own error_page entry.
"error_page.default" to handle all codes which do not have their own error_page
entry.
@@ -81,8 +86,9 @@ Unanticipated errors
CherryPy also has a generic error handling mechanism: whenever an unanticipated
error occurs in your code, it will call
:func:`Request.error_response<cherrypy._cprequest.Request.error_response>` to set
the response status, headers, and body. By default, this is the same output as
:func:`Request.error_response<cherrypy._cprequest.Request.error_response>` to
set the response status, headers, and body. By default, this is the same
output as
:class:`HTTPError(500) <cherrypy._cperror.HTTPError>`. If you want to provide
some other behavior, you generally replace "request.error_response".
@@ -93,40 +99,50 @@ send an e-mail containing the error::
def handle_error():
cherrypy.response.status = 500
cherrypy.response.body = ["<html><body>Sorry, an error occured</body></html>"]
sendMail('error@domain.com', 'Error in your web app', _cperror.format_exc())
cherrypy.response.body = [
"<html><body>Sorry, an error occured</body></html>"
]
sendMail('error@domain.com',
'Error in your web app',
_cperror.format_exc())
class Root:
_cp_config = {'request.error_response': handle_error}
Note that you have to explicitly set :attr:`response.body <cherrypy._cprequest.Response.body>`
Note that you have to explicitly set
:attr:`response.body <cherrypy._cprequest.Response.body>`
and not simply return an error message as a result.
"""
from cgi import escape as _escape
from sys import exc_info as _exc_info
from traceback import format_exception as _format_exception
from cherrypy._cpcompat import basestring, bytestr, iteritems, ntob, tonative, urljoin as _urljoin
from cherrypy._cpcompat import basestring, bytestr, iteritems, ntob
from cherrypy._cpcompat import tonative, urljoin as _urljoin
from cherrypy.lib import httputil as _httputil
class CherryPyException(Exception):
"""A base class for CherryPy exceptions."""
pass
class TimeoutError(CherryPyException):
"""Exception raised when Response.timed_out is detected."""
pass
class InternalRedirect(CherryPyException):
"""Exception raised to switch to the handler for a different URL.
This exception will redirect processing to another path within the site
(without informing the client). Provide the new path as an argument when
raising the exception. Provide any params in the querystring for the new URL.
raising the exception. Provide any params in the querystring for the new
URL.
"""
def __init__(self, path, query_string=""):
@@ -152,6 +168,7 @@ class InternalRedirect(CherryPyException):
class HTTPRedirect(CherryPyException):
"""Exception raised when the request should be redirected.
This exception will force a HTTP redirect to the URL or URL's you give it.
@@ -222,7 +239,8 @@ class HTTPRedirect(CherryPyException):
CherryPyException.__init__(self, abs_urls, status)
def set_response(self):
"""Modify cherrypy.response status, headers, and body to represent self.
"""Modify cherrypy.response status, headers, and body to represent
self.
CherryPy uses this internally, but you can also use it to create an
HTTPRedirect object and set its output without *raising* the exception.
@@ -240,13 +258,16 @@ class HTTPRedirect(CherryPyException):
# "Unless the request method was HEAD, the entity of the response
# SHOULD contain a short hypertext note with a hyperlink to the
# new URI(s)."
msg = {300: "This resource can be found at <a href='%s'>%s</a>.",
301: "This resource has permanently moved to <a href='%s'>%s</a>.",
302: "This resource resides temporarily at <a href='%s'>%s</a>.",
303: "This resource can be found at <a href='%s'>%s</a>.",
307: "This resource has moved temporarily to <a href='%s'>%s</a>.",
}[status]
msgs = [msg % (u, u) for u in self.urls]
msg = {
300: "This resource can be found at ",
301: "This resource has permanently moved to ",
302: "This resource resides temporarily at ",
303: "This resource can be found at ",
307: "This resource has moved temporarily to ",
}[status]
msg += '<a href=%s>%s</a>.'
from xml.sax import saxutils
msgs = [msg % (saxutils.quoteattr(u), u) for u in self.urls]
response.body = ntob("<br />\n".join(msgs), 'utf-8')
# Previous code may have set C-L, so we have to reset it
# (allow finalize to set it).
@@ -311,24 +332,27 @@ def clean_headers(status):
class HTTPError(CherryPyException):
"""Exception used to return an HTTP error code (4xx-5xx) to the client.
This exception can be used to automatically send a response using a http status
code, with an appropriate error page. It takes an optional
This exception can be used to automatically send a response using a
http status code, with an appropriate error page. It takes an optional
``status`` argument (which must be between 400 and 599); it defaults to 500
("Internal Server Error"). It also takes an optional ``message`` argument,
which will be returned in the response body. See
`RFC 2616 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4>`_
`RFC2616 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4>`_
for a complete list of available error codes and when to use them.
Examples::
raise cherrypy.HTTPError(403)
raise cherrypy.HTTPError("403 Forbidden", "You are not allowed to access this resource.")
raise cherrypy.HTTPError(
"403 Forbidden", "You are not allowed to access this resource.")
"""
status = None
"""The HTTP status code. May be of type int or str (with a Reason-Phrase)."""
"""The HTTP status code. May be of type int or str (with a Reason-Phrase).
"""
code = None
"""The integer HTTP status code."""
@@ -352,7 +376,8 @@ class HTTPError(CherryPyException):
CherryPyException.__init__(self, status, message)
def set_response(self):
"""Modify cherrypy.response status, headers, and body to represent self.
"""Modify cherrypy.response status, headers, and body to represent
self.
CherryPy uses this internally, but you can also use it to create an
HTTPError object and set its output without *raising* the exception.
@@ -369,11 +394,11 @@ class HTTPError(CherryPyException):
tb = None
if cherrypy.serving.request.show_tracebacks:
tb = format_exc()
response.headers['Content-Type'] = "text/html;charset=utf-8"
response.headers.pop('Content-Length', None)
content = ntob(self.get_error_page(self.status, traceback=tb,
message=self._message), 'utf-8')
content = self.get_error_page(self.status, traceback=tb,
message=self._message)
response.body = content
_be_ie_unfriendly(self.code)
@@ -387,6 +412,7 @@ class HTTPError(CherryPyException):
class NotFound(HTTPError):
"""Exception raised when a URL could not be mapped to any handler (404).
This is equivalent to raising
@@ -402,7 +428,8 @@ class NotFound(HTTPError):
HTTPError.__init__(self, 404, "The path '%s' was not found." % path)
_HTTPErrorTemplate = '''<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
_HTTPErrorTemplate = '''<!DOCTYPE html PUBLIC
"-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
@@ -425,12 +452,15 @@ _HTTPErrorTemplate = '''<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitiona
<p>%(message)s</p>
<pre id="traceback">%(traceback)s</pre>
<div id="powered_by">
<span>Powered by <a href="http://www.cherrypy.org">CherryPy %(version)s</a></span>
<span>
Powered by <a href="http://www.cherrypy.org">CherryPy %(version)s</a>
</span>
</div>
</body>
</html>
'''
def get_error_page(status, **kwargs):
"""Return an HTML page, containing a pretty error response.
@@ -464,13 +494,33 @@ def get_error_page(status, **kwargs):
# Use a custom template or callable for the error page?
pages = cherrypy.serving.request.error_page
error_page = pages.get(code) or pages.get('default')
# Default template, can be overridden below.
template = _HTTPErrorTemplate
if error_page:
try:
if hasattr(error_page, '__call__'):
return error_page(**kwargs)
# The caller function may be setting headers manually,
# so we delegate to it completely. We may be returning
# an iterator as well as a string here.
#
# We *must* make sure any content is not unicode.
result = error_page(**kwargs)
if cherrypy.lib.is_iterator(result):
from cherrypy.lib.encoding import UTF8StreamEncoder
return UTF8StreamEncoder(result)
elif isinstance(result, cherrypy._cpcompat.unicodestr):
return result.encode('utf-8')
else:
if not isinstance(result, cherrypy._cpcompat.bytestr):
raise ValueError('error page function did not '
'return a bytestring, unicodestring or an '
'iterator - returned object of type %s.'
% (type(result).__name__))
return result
else:
data = open(error_page, 'rb').read()
return tonative(data) % kwargs
# Load the template from this path.
template = tonative(open(error_page, 'rb').read())
except:
e = _format_exception(*_exc_info())[-1]
m = kwargs['message']
@@ -479,14 +529,18 @@ def get_error_page(status, **kwargs):
m += "In addition, the custom error page failed:\n<br />%s" % e
kwargs['message'] = m
return _HTTPErrorTemplate % kwargs
response = cherrypy.serving.response
response.headers['Content-Type'] = "text/html;charset=utf-8"
result = template % kwargs
return result.encode('utf-8')
_ie_friendly_error_sizes = {
400: 512, 403: 256, 404: 512, 405: 256,
406: 512, 408: 512, 409: 512, 410: 256,
500: 512, 501: 512, 505: 512,
}
}
def _be_ie_unfriendly(status):
@@ -525,6 +579,7 @@ def format_exc(exc=None):
finally:
del exc
def bare_error(extrabody=None):
"""Produce status, headers, body for a critical error.
@@ -550,7 +605,5 @@ def bare_error(extrabody=None):
return (ntob("500 Internal Server Error"),
[(ntob('Content-Type'), ntob('text/plain')),
(ntob('Content-Length'), ntob(str(len(body)),'ISO-8859-1'))],
(ntob('Content-Length'), ntob(str(len(body)), 'ISO-8859-1'))],
[body])

55
lib/cherrypy/_cplogging.py
View File

@@ -34,10 +34,11 @@ and another set of rules specific to each application. The global log
manager is found at :func:`cherrypy.log`, and the log manager for each
application is found at :attr:`app.log<cherrypy._cptree.Application.log>`.
If you're inside a request, the latter is reachable from
``cherrypy.request.app.log``; if you're outside a request, you'll have to obtain
a reference to the ``app``: either the return value of
``cherrypy.request.app.log``; if you're outside a request, you'll have to
obtain a reference to the ``app``: either the return value of
:func:`tree.mount()<cherrypy._cptree.Tree.mount>` or, if you used
:func:`quickstart()<cherrypy.quickstart>` instead, via ``cherrypy.tree.apps['/']``.
:func:`quickstart()<cherrypy.quickstart>` instead, via
``cherrypy.tree.apps['/']``.
By default, the global logs are named "cherrypy.error" and "cherrypy.access",
and the application logs are named "cherrypy.error.2378745" and
@@ -55,6 +56,13 @@ errors! The format of access messages is highly formalized, but the error log
isn't--it receives messages from a variety of sources (including full error
tracebacks, if enabled).
If you are logging the access log and error log to the same source, then there
is a possibility that a specially crafted error message may replicate an access
log message as described in CWE-117. In this case it is the application
developer's responsibility to manually escape data before using CherryPy's log()
functionality, or they may create an application that is vulnerable to CWE-117.
This would be achieved by using a custom handler escape any special characters,
and attached as described below.
Custom Handlers
===============
@@ -113,6 +121,7 @@ from cherrypy._cpcompat import ntob, py3k
class NullHandler(logging.Handler):
"""A no-op logging handler to silence the logging.lastResort handler."""
def handle(self, record):
@@ -126,6 +135,7 @@ class NullHandler(logging.Handler):
class LogManager(object):
"""An object to assist both simple and advanced logging.
``cherrypy.log`` is an instance of this class.
@@ -166,8 +176,10 @@ class LogManager(object):
self.error_log = logging.getLogger("%s.error" % logger_root)
self.access_log = logging.getLogger("%s.access" % logger_root)
else:
self.error_log = logging.getLogger("%s.error.%s" % (logger_root, appid))
self.access_log = logging.getLogger("%s.access.%s" % (logger_root, appid))
self.error_log = logging.getLogger(
"%s.error.%s" % (logger_root, appid))
self.access_log = logging.getLogger(
"%s.access.%s" % (logger_root, appid))
self.error_log.setLevel(logging.INFO)
self.access_log.setLevel(logging.INFO)
@@ -187,7 +199,8 @@ class LogManager(object):
h.stream = open(h.baseFilename, h.mode)
h.release()
def error(self, msg='', context='', severity=logging.INFO, traceback=False):
def error(self, msg='', context='', severity=logging.INFO,
traceback=False):
"""Write the given ``msg`` to the error log.
This is not just for errors! Applications may call this at any time
@@ -207,8 +220,9 @@ class LogManager(object):
def access(self):
"""Write to the access log (in Apache/NCSA Combined Log format).
See http://httpd.apache.org/docs/2.0/logs.html#combined for format
details.
See the
`apache documentation <http://httpd.apache.org/docs/current/logs.html#combined>`_
for format details.
CherryPy calls this automatically for you. Note there are no arguments;
it collects the data itself from
@@ -242,6 +256,7 @@ class LogManager(object):
'b': dict.get(outheaders, 'Content-Length', '') or "-",
'f': dict.get(inheaders, 'Referer', ''),
'a': dict.get(inheaders, 'User-Agent', ''),
'o': dict.get(inheaders, 'Host', '-'),
}
if py3k:
for k, v in atoms.items():
@@ -261,7 +276,8 @@ class LogManager(object):
atoms[k] = v
try:
self.access_log.log(logging.INFO, self.access_log_format.format(**atoms))
self.access_log.log(
logging.INFO, self.access_log_format.format(**atoms))
except:
self(traceback=True)
else:
@@ -277,7 +293,8 @@ class LogManager(object):
atoms[k] = v.replace('"', '\\"')
try:
self.access_log.log(logging.INFO, self.access_log_format % atoms)
self.access_log.log(
logging.INFO, self.access_log_format % atoms)
except:
self(traceback=True)
@@ -295,15 +312,13 @@ class LogManager(object):
if getattr(h, "_cpbuiltin", None) == key:
return h
# ------------------------- Screen handlers ------------------------- #
def _set_screen_handler(self, log, enable, stream=None):
h = self._get_builtin_handler(log, "screen")
if enable:
if not h:
if stream is None:
stream=sys.stderr
stream = sys.stderr
h = logging.StreamHandler(stream)
h.setFormatter(logfmt)
h._cpbuiltin = "screen"
@@ -320,7 +335,7 @@ class LogManager(object):
self._set_screen_handler(self.error_log, newvalue, stream=sys.stderr)
self._set_screen_handler(self.access_log, newvalue, stream=sys.stdout)
screen = property(_get_screen, _set_screen,
doc="""Turn stderr/stdout logging on or off.
doc="""Turn stderr/stdout logging on or off.
If you set this to True, it'll add the appropriate StreamHandler for
you. If you set it to False, it will remove the handler.
@@ -354,10 +369,11 @@ class LogManager(object):
if h:
return h.baseFilename
return ''
def _set_error_file(self, newvalue):
self._set_file_handler(self.error_log, newvalue)
error_file = property(_get_error_file, _set_error_file,
doc="""The filename for self.error_log.
doc="""The filename for self.error_log.
If you set this to a string, it'll add the appropriate FileHandler for
you. If you set it to ``None`` or ``''``, it will remove the handler.
@@ -368,10 +384,11 @@ class LogManager(object):
if h:
return h.baseFilename
return ''
def _set_access_file(self, newvalue):
self._set_file_handler(self.access_log, newvalue)
access_file = property(_get_access_file, _set_access_file,
doc="""The filename for self.access_log.
doc="""The filename for self.access_log.
If you set this to a string, it'll add the appropriate FileHandler for
you. If you set it to ``None`` or ``''``, it will remove the handler.
@@ -396,7 +413,7 @@ class LogManager(object):
def _set_wsgi(self, newvalue):
self._set_wsgi_handler(self.error_log, newvalue)
wsgi = property(_get_wsgi, _set_wsgi,
doc="""Write errors to wsgi.errors.
doc="""Write errors to wsgi.errors.
If you set this to True, it'll add the appropriate
:class:`WSGIErrorHandler<cherrypy._cplogging.WSGIErrorHandler>` for you
@@ -406,6 +423,7 @@ class LogManager(object):
class WSGIErrorHandler(logging.Handler):
"A handler class which writes logging records to environ['wsgi.errors']."
def flush(self):
@@ -428,7 +446,8 @@ class WSGIErrorHandler(logging.Handler):
msg = self.format(record)
fs = "%s\n"
import types
if not hasattr(types, "UnicodeType"): #if no unicode support...
# if no unicode support...
if not hasattr(types, "UnicodeType"):
stream.write(fs % msg)
else:
try:

51
lib/cherrypy/_cpmodpy.py
View File

@@ -35,11 +35,11 @@ Listen 8080
LoadModule python_module /usr/lib/apache2/modules/mod_python.so
<Location "/">
PythonPath "sys.path+['/path/to/my/application']"
SetHandler python-program
PythonHandler cherrypy._cpmodpy::handler
PythonOption cherrypy.setup myapp::setup_server
PythonDebug On
PythonPath "sys.path+['/path/to/my/application']"
SetHandler python-program
PythonHandler cherrypy._cpmodpy::handler
PythonOption cherrypy.setup myapp::setup_server
PythonDebug On
</Location>
# End
@@ -67,11 +67,11 @@ from cherrypy.lib import httputil
# ------------------------------ Request-handling
def setup(req):
from mod_python import apache
# Run any setup functions defined by a "PythonOption cherrypy.setup" directive.
# Run any setup functions defined by a "PythonOption cherrypy.setup"
# directive.
options = req.get_options()
if 'cherrypy.setup' in options:
for function in options['cherrypy.setup'].split():
@@ -106,7 +106,7 @@ def setup(req):
elif logging.WARNING >= level:
newlevel = apache.APLOG_WARNING
# On Windows, req.server is required or the msg will vanish. See
# http://www.modpython.org/pipermail/mod_python/2003-October/014291.html.
# http://www.modpython.org/pipermail/mod_python/2003-October/014291.html
# Also, "When server is not specified...LogLevel does not apply..."
apache.log_error(msg, newlevel, req.server)
engine.subscribe('log', _log)
@@ -124,6 +124,7 @@ def setup(req):
class _ReadOnlyRequest:
expose = ('read', 'readline', 'readlines')
def __init__(self, req):
for method in self.expose:
self.__dict__[method] = getattr(req, method)
@@ -132,6 +133,8 @@ class _ReadOnlyRequest:
recursive = False
_isSetUp = False
def handler(req):
from mod_python import apache
try:
@@ -142,9 +145,11 @@ def handler(req):
# Obtain a Request object from CherryPy
local = req.connection.local_addr
local = httputil.Host(local[0], local[1], req.connection.local_host or "")
local = httputil.Host(
local[0], local[1], req.connection.local_host or "")
remote = req.connection.remote_addr
remote = httputil.Host(remote[0], remote[1], req.connection.remote_host or "")
remote = httputil.Host(
remote[0], remote[1], req.connection.remote_host or "")
scheme = req.parsed_uri[0] or 'http'
req.get_basic_auth_pw()
@@ -210,10 +215,12 @@ def handler(req):
if not recursive:
if ir.path in redirections:
raise RuntimeError("InternalRedirector visited the "
"same URL twice: %r" % ir.path)
raise RuntimeError(
"InternalRedirector visited the same URL "
"twice: %r" % ir.path)
else:
# Add the *previous* path_info + qs to redirections.
# Add the *previous* path_info + qs to
# redirections.
if qs:
qs = "?" + qs
redirections.append(sn + path + qs)
@@ -224,8 +231,9 @@ def handler(req):
qs = ir.query_string
rfile = BytesIO()
send_response(req, response.output_status, response.header_list,
response.body, response.stream)
send_response(
req, response.output_status, response.header_list,
response.body, response.stream)
finally:
app.release_serving()
except:
@@ -260,14 +268,12 @@ def send_response(req, status, headers, body, stream=False):
req.write(seg)
# --------------- Startup tools for CherryPy + mod_python --------------- #
import os
import re
try:
import subprocess
def popen(fullcmd):
p = subprocess.Popen(fullcmd, shell=True,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
@@ -284,8 +290,12 @@ def read_process(cmd, args=""):
pipeout = popen(fullcmd)
try:
firstline = pipeout.readline()
if (re.search(ntob("(not recognized|No such file|not found)"), firstline,
re.IGNORECASE)):
cmd_not_found = re.search(
ntob("(not recognized|No such file|not found)"),
firstline,
re.IGNORECASE
)
if cmd_not_found:
raise IOError('%s must be on your system path.' % cmd)
output = firstline + pipeout.read()
finally:
@@ -341,4 +351,3 @@ LoadModule python_module modules/mod_python.so
def stop(self):
os.popen("apache -k stop")
self.ready = False

25
lib/cherrypy/_cpnative_server.py
View File

@@ -46,9 +46,11 @@ class NativeGateway(wsgiserver.Gateway):
request.app = app
request.prev = prev
# Run the CherryPy Request object and obtain the response
# Run the CherryPy Request object and obtain the
# response
try:
request.run(method, path, qs, req.request_protocol, headers, rfile)
request.run(method, path, qs,
req.request_protocol, headers, rfile)
break
except cherrypy.InternalRedirect:
ir = sys.exc_info()[1]
@@ -57,10 +59,12 @@ class NativeGateway(wsgiserver.Gateway):
if not self.recursive:
if ir.path in redirections:
raise RuntimeError("InternalRedirector visited the "
"same URL twice: %r" % ir.path)
raise RuntimeError(
"InternalRedirector visited the same "
"URL twice: %r" % ir.path)
else:
# Add the *previous* path_info + qs to redirections.
# Add the *previous* path_info + qs to
# redirections.
if qs:
qs = "?" + qs
redirections.append(sn + path + qs)
@@ -78,7 +82,7 @@ class NativeGateway(wsgiserver.Gateway):
app.release_serving()
except:
tb = format_exc()
#print tb
# print tb
cherrypy.log(tb, 'NATIVE_ADAPTER', severity=logging.ERROR)
s, h, b = bare_error()
self.send_response(s, h, b)
@@ -102,6 +106,7 @@ class NativeGateway(wsgiserver.Gateway):
class CPHTTPServer(wsgiserver.HTTPServer):
"""Wrapper for wsgiserver.HTTPServer.
wsgiserver has been designed to not reference CherryPy in any way,
@@ -123,8 +128,10 @@ class CPHTTPServer(wsgiserver.HTTPServer):
maxthreads=server_adapter.thread_pool_max,
server_name=server_name)
self.max_request_header_size = self.server_adapter.max_request_header_size or 0
self.max_request_body_size = self.server_adapter.max_request_body_size or 0
self.max_request_header_size = (
self.server_adapter.max_request_header_size or 0)
self.max_request_body_size = (
self.server_adapter.max_request_body_size or 0)
self.request_queue_size = self.server_adapter.socket_queue_size
self.timeout = self.server_adapter.socket_timeout
self.shutdown_timeout = self.server_adapter.shutdown_timeout
@@ -145,5 +152,3 @@ class CPHTTPServer(wsgiserver.HTTPServer):
self.server_adapter.ssl_certificate,
self.server_adapter.ssl_private_key,
self.server_adapter.ssl_certificate_chain)

180
lib/cherrypy/_cpreqbody.py
View File

@@ -3,8 +3,10 @@
.. versionadded:: 3.2
Application authors have complete control over the parsing of HTTP request
entities. In short, :attr:`cherrypy.request.body<cherrypy._cprequest.Request.body>`
is now always set to an instance of :class:`RequestBody<cherrypy._cpreqbody.RequestBody>`,
entities. In short,
:attr:`cherrypy.request.body<cherrypy._cprequest.Request.body>`
is now always set to an instance of
:class:`RequestBody<cherrypy._cpreqbody.RequestBody>`,
and *that* class is a subclass of :class:`Entity<cherrypy._cpreqbody.Entity>`.
When an HTTP request includes an entity body, it is often desirable to
@@ -21,9 +23,9 @@ key to look up a value in the
:attr:`request.body.processors<cherrypy._cpreqbody.Entity.processors>` dict.
If the full media
type is not found, then the major type is tried; for example, if no processor
is found for the 'image/jpeg' type, then we look for a processor for the 'image'
types altogether. If neither the full type nor the major type has a matching
processor, then a default processor is used
is found for the 'image/jpeg' type, then we look for a processor for the
'image' types altogether. If neither the full type nor the major type has a
matching processor, then a default processor is used
(:func:`default_proc<cherrypy._cpreqbody.Entity.default_proc>`). For most
types, this means no processing is done, and the body is left unread as a
raw byte stream. Processors are configurable in an 'on_start_resource' hook.
@@ -74,31 +76,36 @@ Here's the built-in JSON tool for an example::
415, 'Expected an application/json content type')
request.body.processors['application/json'] = json_processor
We begin by defining a new ``json_processor`` function to stick in the ``processors``
dictionary. All processor functions take a single argument, the ``Entity`` instance
they are to process. It will be called whenever a request is received (for those
URI's where the tool is turned on) which has a ``Content-Type`` of
"application/json".
We begin by defining a new ``json_processor`` function to stick in the
``processors`` dictionary. All processor functions take a single argument,
the ``Entity`` instance they are to process. It will be called whenever a
request is received (for those URI's where the tool is turned on) which
has a ``Content-Type`` of "application/json".
First, it checks for a valid ``Content-Length`` (raising 411 if not valid), then
reads the remaining bytes on the socket. The ``fp`` object knows its own length, so
it won't hang waiting for data that never arrives. It will return when all data
has been read. Then, we decode those bytes using Python's built-in ``json`` module,
and stick the decoded result onto ``request.json`` . If it cannot be decoded, we
raise 400.
First, it checks for a valid ``Content-Length`` (raising 411 if not valid),
then reads the remaining bytes on the socket. The ``fp`` object knows its
own length, so it won't hang waiting for data that never arrives. It will
return when all data has been read. Then, we decode those bytes using
Python's built-in ``json`` module, and stick the decoded result onto
``request.json`` . If it cannot be decoded, we raise 400.
If the "force" argument is True (the default), the ``Tool`` clears the ``processors``
dict so that request entities of other ``Content-Types`` aren't parsed at all. Since
there's no entry for those invalid MIME types, the ``default_proc`` method of ``cherrypy.request.body``
is called. But this does nothing by default (usually to provide the page handler an opportunity to handle it.)
But in our case, we want to raise 415, so we replace ``request.body.default_proc``
If the "force" argument is True (the default), the ``Tool`` clears the
``processors`` dict so that request entities of other ``Content-Types``
aren't parsed at all. Since there's no entry for those invalid MIME
types, the ``default_proc`` method of ``cherrypy.request.body`` is
called. But this does nothing by default (usually to provide the page
handler an opportunity to handle it.)
But in our case, we want to raise 415, so we replace
``request.body.default_proc``
with the error (``HTTPError`` instances, when called, raise themselves).
If we were defining a custom processor, we can do so without making a ``Tool``. Just add the config entry::
If we were defining a custom processor, we can do so without making a ``Tool``.
Just add the config entry::
request.body.processors = {'application/json': json_processor}
Note that you can only replace the ``processors`` dict wholesale this way, not update the existing one.
Note that you can only replace the ``processors`` dict wholesale this way,
not update the existing one.
"""
try:
@@ -129,7 +136,7 @@ from cherrypy._cpcompat import basestring, ntob, ntou
from cherrypy.lib import httputil
# -------------------------------- Processors -------------------------------- #
# ------------------------------- Processors -------------------------------- #
def process_urlencoded(entity):
"""Read application/x-www-form-urlencoded data into entity.params."""
@@ -209,8 +216,10 @@ def process_multipart(entity):
if part.fp.done:
break
def process_multipart_form_data(entity):
"""Read all multipart/form-data parts into entity.parts or entity.params."""
"""Read all multipart/form-data parts into entity.parts or entity.params.
"""
process_multipart(entity)
kept_parts = []
@@ -235,6 +244,7 @@ def process_multipart_form_data(entity):
entity.parts = kept_parts
def _old_process_multipart(entity):
"""The behavior of 3.2 and lower. Deprecated and will be changed in 3.3."""
process_multipart(entity)
@@ -263,11 +273,9 @@ def _old_process_multipart(entity):
params[key] = value
# --------------------------------- Entities --------------------------------- #
# -------------------------------- Entities --------------------------------- #
class Entity(object):
"""An HTTP request body, or MIME multipart body.
This class collects information about the HTTP request entity. When a
@@ -277,16 +285,19 @@ class Entity(object):
Between the ``before_request_body`` and ``before_handler`` tools, CherryPy
tries to process the request body (if any) by calling
:func:`request.body.process<cherrypy._cpreqbody.RequestBody.process`.
This uses the ``content_type`` of the Entity to look up a suitable processor
in :attr:`Entity.processors<cherrypy._cpreqbody.Entity.processors>`, a dict.
:func:`request.body.process<cherrypy._cpreqbody.RequestBody.process>`.
This uses the ``content_type`` of the Entity to look up a suitable
processor in
:attr:`Entity.processors<cherrypy._cpreqbody.Entity.processors>`,
a dict.
If a matching processor cannot be found for the complete Content-Type,
it tries again using the major type. For example, if a request with an
entity of type "image/jpeg" arrives, but no processor can be found for
that complete type, then one is sought for the major type "image". If a
processor is still not found, then the
:func:`default_proc<cherrypy._cpreqbody.Entity.default_proc>` method of the
Entity is called (which does nothing by default; you can override this too).
:func:`default_proc<cherrypy._cpreqbody.Entity.default_proc>` method
of the Entity is called (which does nothing by default; you can
override this too).
CherryPy includes processors for the "application/x-www-form-urlencoded"
type, the "multipart/form-data" type, and the "multipart" major type.
@@ -381,7 +392,8 @@ class Entity(object):
"""A dict of Content-Type names to processor methods."""
parts = None
"""A list of Part instances if ``Content-Type`` is of major type "multipart"."""
"""A list of Part instances if ``Content-Type`` is of major type
"multipart"."""
part_class = None
"""The class used for multipart parts.
@@ -414,7 +426,8 @@ class Entity(object):
self.content_type = httputil.HeaderElement.from_str(
self.default_content_type)
# Copy the class 'attempt_charsets', prepending any Content-Type charset
# Copy the class 'attempt_charsets', prepending any Content-Type
# charset
dec = self.content_type.params.get("charset", None)
if dec:
self.attempt_charsets = [dec] + [c for c in self.attempt_charsets
@@ -426,7 +439,10 @@ class Entity(object):
self.length = None
clen = headers.get('Content-Length', None)
# If Transfer-Encoding is 'chunked', ignore any Content-Length.
if clen is not None and 'chunked' not in headers.get('Transfer-Encoding', ''):
if (
clen is not None and
'chunked' not in headers.get('Transfer-Encoding', '')
):
try:
self.length = int(clen)
except ValueError:
@@ -444,12 +460,18 @@ class Entity(object):
self.name = self.name[1:-1]
if 'filename' in disp.params:
self.filename = disp.params['filename']
if self.filename.startswith('"') and self.filename.endswith('"'):
if (
self.filename.startswith('"') and
self.filename.endswith('"')
):
self.filename = self.filename[1:-1]
# The 'type' attribute is deprecated in 3.2; remove it in 3.3.
type = property(lambda self: self.content_type,
doc="""A deprecated alias for :attr:`content_type<cherrypy._cpreqbody.Entity.content_type>`.""")
type = property(
lambda self: self.content_type,
doc="A deprecated alias for "
":attr:`content_type<cherrypy._cpreqbody.Entity.content_type>`."
)
def read(self, size=None, fp_out=None):
return self.fp.read(size, fp_out)
@@ -473,7 +495,10 @@ class Entity(object):
return self.__next__()
def read_into_file(self, fp_out=None):
"""Read the request body into fp_out (or make_file() if None). Return fp_out."""
"""Read the request body into fp_out (or make_file() if None).
Return fp_out.
"""
if fp_out is None:
fp_out = self.make_file()
self.read(fp_out=fp_out)
@@ -515,7 +540,9 @@ class Entity(object):
proc(self)
def default_proc(self):
"""Called if a more-specific processor is not found for the ``Content-Type``."""
"""Called if a more-specific processor is not found for the
``Content-Type``.
"""
# Leave the fp alone for someone else to read. This works fine
# for request.body, but the Part subclasses need to override this
# so they can move on to the next part.
@@ -523,6 +550,7 @@ class Entity(object):
class Part(Entity):
"""A MIME part entity, part of a multipart entity."""
# "The default character set, which must be assumed in the absence of a
@@ -554,10 +582,11 @@ class Part(Entity):
# This is the default in stdlib cgi. We may want to increase it.
maxrambytes = 1000
"""The threshold of bytes after which point the ``Part`` will store its data
in a file (generated by :func:`make_file<cherrypy._cprequest.Entity.make_file>`)
instead of a string. Defaults to 1000, just like the :mod:`cgi` module in
Python's standard library.
"""The threshold of bytes after which point the ``Part`` will store
its data in a file (generated by
:func:`make_file<cherrypy._cprequest.Entity.make_file>`)
instead of a string. Defaults to 1000, just like the :mod:`cgi`
module in Python's standard library.
"""
def __init__(self, fp, headers, boundary):
@@ -607,9 +636,9 @@ class Part(Entity):
If the 'fp_out' argument is None (the default), all bytes read are
returned in a single byte string.
If the 'fp_out' argument is not None, it must be a file-like object that
supports the 'write' method; all bytes read will be written to the fp,
and that fp is returned.
If the 'fp_out' argument is not None, it must be a file-like
object that supports the 'write' method; all bytes read will be
written to the fp, and that fp is returned.
"""
endmarker = self.boundary + ntob("--")
delim = ntob("")
@@ -617,7 +646,7 @@ class Part(Entity):
lines = []
seen = 0
while True:
line = self.fp.readline(1<<16)
line = self.fp.readline(1 << 16)
if not line:
raise EOFError("Illegal end of multipart body.")
if line.startswith(ntob("--")) and prev_lf:
@@ -664,14 +693,18 @@ class Part(Entity):
return result
else:
raise cherrypy.HTTPError(
400, "The request entity could not be decoded. The following "
"charsets were attempted: %s" % repr(self.attempt_charsets))
400,
"The request entity could not be decoded. The following "
"charsets were attempted: %s" % repr(self.attempt_charsets)
)
else:
fp_out.seek(0)
return fp_out
def default_proc(self):
"""Called if a more-specific processor is not found for the ``Content-Type``."""
"""Called if a more-specific processor is not found for the
``Content-Type``.
"""
if self.filename:
# Always read into a file if a .filename was given.
self.file = self.read_into_file()
@@ -683,7 +716,10 @@ class Part(Entity):
self.file = result
def read_into_file(self, fp_out=None):
"""Read the request body into fp_out (or make_file() if None). Return fp_out."""
"""Read the request body into fp_out (or make_file() if None).
Return fp_out.
"""
if fp_out is None:
fp_out = self.make_file()
self.read_lines_to_boundary(fp_out=fp_out)
@@ -696,23 +732,30 @@ try:
except ValueError:
# Python 2.4 and lower
class Infinity(object):
def __cmp__(self, other):
return 1
def __sub__(self, other):
return self
inf = Infinity()
comma_separated_headers = ['Accept', 'Accept-Charset', 'Accept-Encoding',
'Accept-Language', 'Accept-Ranges', 'Allow', 'Cache-Control', 'Connection',
'Content-Encoding', 'Content-Language', 'Expect', 'If-Match',
'If-None-Match', 'Pragma', 'Proxy-Authenticate', 'Te', 'Trailer',
'Transfer-Encoding', 'Upgrade', 'Vary', 'Via', 'Warning', 'Www-Authenticate']
comma_separated_headers = [
'Accept', 'Accept-Charset', 'Accept-Encoding',
'Accept-Language', 'Accept-Ranges', 'Allow',
'Cache-Control', 'Connection', 'Content-Encoding',
'Content-Language', 'Expect', 'If-Match',
'If-None-Match', 'Pragma', 'Proxy-Authenticate',
'Te', 'Trailer', 'Transfer-Encoding', 'Upgrade',
'Vary', 'Via', 'Warning', 'Www-Authenticate'
]
class SizedReader:
def __init__(self, fp, length, maxbytes, bufsize=DEFAULT_BUFFER_SIZE, has_trailers=False):
def __init__(self, fp, length, maxbytes, bufsize=DEFAULT_BUFFER_SIZE,
has_trailers=False):
# Wrap our fp in a buffer so peek() works
self.fp = fp
self.length = length
@@ -736,9 +779,9 @@ class SizedReader:
If the 'fp_out' argument is None (the default), all bytes read are
returned in a single byte string.
If the 'fp_out' argument is not None, it must be a file-like object that
supports the 'write' method; all bytes read will be written to the fp,
and None is returned.
If the 'fp_out' argument is not None, it must be a file-like
object that supports the 'write' method; all bytes read will be
written to the fp, and None is returned.
"""
if self.length is None:
@@ -889,13 +932,15 @@ class SizedReader:
class RequestBody(Entity):
"""The entity of the HTTP request."""
bufsize = 8 * 1024
"""The buffer size used when reading the socket."""
# Don't parse the request body at all if the client didn't provide
# a Content-Type header. See http://www.cherrypy.org/ticket/790
# a Content-Type header. See
# https://bitbucket.org/cherrypy/cherrypy/issue/790
default_content_type = ''
"""This defines a default ``Content-Type`` to use if no Content-Type header
is given. The empty string is used for RequestBody, which results in the
@@ -907,7 +952,9 @@ class RequestBody(Entity):
"""
maxbytes = None
"""Raise ``MaxSizeExceeded`` if more bytes than this are read from the socket."""
"""Raise ``MaxSizeExceeded`` if more bytes than this are read from
the socket.
"""
def __init__(self, fp, headers, params=None, request_params=None):
Entity.__init__(self, fp, headers, params)
@@ -952,7 +999,8 @@ class RequestBody(Entity):
# add them in here.
request_params = self.request_params
for key, value in self.params.items():
# Python 2 only: keyword arguments must be byte strings (type 'str').
# Python 2 only: keyword arguments must be byte strings (type
# 'str').
if sys.version_info < (3, 0):
if isinstance(key, unicode):
key = key.encode('ISO-8859-1')

57
lib/cherrypy/_cprequest.py
View File

@@ -13,6 +13,7 @@ from cherrypy.lib import httputil, file_generator
class Hook(object):
"""A callback and its metadata: failsafe, priority, and kwargs."""
callback = None
@@ -71,6 +72,7 @@ class Hook(object):
class HookMap(dict):
"""A map of call points to lists of callbacks (Hook objects)."""
def __new__(cls, points=None):
@@ -122,7 +124,11 @@ class HookMap(dict):
def __repr__(self):
cls = self.__class__
return "%s.%s(points=%r)" % (cls.__module__, cls.__name__, copykeys(self))
return "%s.%s(points=%r)" % (
cls.__module__,
cls.__name__,
copykeys(self)
)
# Config namespace handlers
@@ -139,14 +145,17 @@ def hooks_namespace(k, v):
v = Hook(v)
cherrypy.serving.request.hooks[hookpoint].append(v)
def request_namespace(k, v):
"""Attach request attributes declared in config."""
# Provides config entries to set request.body attrs (like attempt_charsets).
# Provides config entries to set request.body attrs (like
# attempt_charsets).
if k[:5] == 'body.':
setattr(cherrypy.serving.request.body, k[5:], v)
else:
setattr(cherrypy.serving.request, k, v)
def response_namespace(k, v):
"""Attach response attributes declared in config."""
# Provides config entries to set default response headers
@@ -156,6 +165,7 @@ def response_namespace(k, v):
else:
setattr(cherrypy.serving.response, k, v)
def error_page_namespace(k, v):
"""Attach error pages declared in config."""
if k != 'default':
@@ -170,6 +180,7 @@ hookpoints = ['on_start_resource', 'before_request_body',
class Request(object):
"""An HTTP request.
This object represents the metadata of an HTTP request message;
@@ -304,7 +315,10 @@ class Request(object):
methods_with_bodies = ("POST", "PUT")
"""
A sequence of HTTP methods for which CherryPy will automatically
attempt to read a body from the rfile."""
attempt to read a body from the rfile. If you are going to change
this property, modify it on the configuration (recommended)
or on the "hook point" `on_start_resource`.
"""
body = None
"""
@@ -419,9 +433,9 @@ class Request(object):
If a callable is provided, it will be called by default with keyword
arguments 'status', 'message', 'traceback', and 'version', as for a
string-formatting template. The callable must return a string or iterable of
strings which will be set to response.body. It may also override headers or
perform any other processing.
string-formatting template. The callable must return a string or
iterable of strings which will be set to response.body. It may also
override headers or perform any other processing.
If no entry is given for an error code, and no 'default' entry exists,
a default template will be used.
@@ -704,9 +718,10 @@ class Request(object):
name = name.title()
value = value.strip()
# Warning: if there is more than one header entry for cookies (AFAIK,
# only Konqueror does that), only the last one will remain in headers
# (but they will be correctly stored in request.cookie).
# Warning: if there is more than one header entry for cookies
# (AFAIK, only Konqueror does that), only the last one will
# remain in headers (but they will be correctly stored in
# request.cookie).
if "=?" in value:
dict.__setitem__(headers, name, httputil.decode_TEXT(value))
else:
@@ -738,7 +753,8 @@ class Request(object):
# First, see if there is a custom dispatch at this URI. Custom
# dispatchers can only be specified in app.config, not in _cp_config
# (since custom dispatchers may not even have an app.root).
dispatch = self.app.find_config(path, "request.dispatch", self.dispatch)
dispatch = self.app.find_config(
path, "request.dispatch", self.dispatch)
# dispatch() should set self.handler and self.config
dispatch(path)
@@ -760,13 +776,13 @@ class Request(object):
def _get_body_params(self):
warnings.warn(
"body_params is deprecated in CherryPy 3.2, will be removed in "
"CherryPy 3.3.",
DeprecationWarning
)
"body_params is deprecated in CherryPy 3.2, will be removed in "
"CherryPy 3.3.",
DeprecationWarning
)
return self.body.params
body_params = property(_get_body_params,
doc= """
doc="""
If the request Content-Type is 'application/x-www-form-urlencoded' or
multipart, this will be a dict of the params pulled from the entity
body; that is, it will be the portion of request.params that come
@@ -780,6 +796,7 @@ class Request(object):
class ResponseBody(object):
"""The body of the HTTP response (the response entity)."""
if py3k:
@@ -822,6 +839,7 @@ class ResponseBody(object):
class Response(object):
"""An HTTP Response, including status, headers, and body."""
status = ""
@@ -889,7 +907,8 @@ class Response(object):
newbody = []
for chunk in self.body:
if py3k and not isinstance(chunk, bytes):
raise TypeError("Chunk %s is not of type 'bytes'." % repr(chunk))
raise TypeError("Chunk %s is not of type 'bytes'." %
repr(chunk))
newbody.append(chunk)
newbody = ntob('').join(newbody)
@@ -906,7 +925,8 @@ class Response(object):
headers = self.headers
self.status = "%s %s" % (code, reason)
self.output_status = ntob(str(code), 'ascii') + ntob(" ") + headers.encode(reason)
self.output_status = ntob(str(code), 'ascii') + \
ntob(" ") + headers.encode(reason)
if self.stream:
# The upshot: wsgiserver will chunk the response if
@@ -951,6 +971,3 @@ class Response(object):
"""
if time.time() > self.time + self.timeout:
self.timed_out = True

55
lib/cherrypy/_cpserver.py
View File

@@ -12,6 +12,7 @@ from cherrypy.process.servers import *
class Server(ServerAdapter):
"""An adapter for an HTTP server.
You can set attributes (like socket_host and socket_port)
@@ -26,15 +27,19 @@ class Server(ServerAdapter):
"""The TCP port on which to listen for connections."""
_socket_host = '127.0.0.1'
def _get_socket_host(self):
return self._socket_host
def _set_socket_host(self, value):
if value == '':
raise ValueError("The empty string ('') is not an allowed value. "
"Use '0.0.0.0' instead to listen on all active "
"interfaces (INADDR_ANY).")
self._socket_host = value
socket_host = property(_get_socket_host, _set_socket_host,
socket_host = property(
_get_socket_host,
_set_socket_host,
doc="""The hostname or IP address on which to listen for connections.
Host values may be any IPv4 or IPv6 address, or any valid hostname.
@@ -56,6 +61,14 @@ class Server(ServerAdapter):
socket_timeout = 10
"""The timeout in seconds for accepted connections (default 10)."""
accepted_queue_size = -1
"""The maximum number of requests which will be queued up before
the server refuses to accept it (default -1, meaning no limit)."""
accepted_queue_timeout = 10
"""The timeout in seconds for attempting to add a request to the
queue when the queue is full (default 10)."""
shutdown_timeout = 5
"""The time to wait for HTTP worker threads to clean up."""
@@ -69,11 +82,13 @@ class Server(ServerAdapter):
"""The number of worker threads to start up in the pool."""
thread_pool_max = -1
"""The maximum size of the worker-thread pool. Use -1 to indicate no limit."""
"""The maximum size of the worker-thread pool. Use -1 to indicate no limit.
"""
max_request_header_size = 500 * 1024
"""The maximum number of bytes allowable in the request headers. If exceeded,
the HTTP server should return "413 Request Entity Too Large"."""
"""The maximum number of bytes allowable in the request headers.
If exceeded, the HTTP server should return "413 Request Entity Too Large".
"""
max_request_body_size = 100 * 1024 * 1024
"""The maximum number of bytes allowable in the request body. If exceeded,
@@ -100,17 +115,19 @@ class Server(ServerAdapter):
if py3k:
ssl_module = 'builtin'
"""The name of a registered SSL adaptation module to use with the builtin
WSGI server. Builtin options are: 'builtin' (to use the SSL library built
into recent versions of Python). You may also register your
own classes in the wsgiserver.ssl_adapters dict."""
"""The name of a registered SSL adaptation module to use with
the builtin WSGI server. Builtin options are: 'builtin' (to
use the SSL library built into recent versions of Python).
You may also register your own classes in the
wsgiserver.ssl_adapters dict."""
else:
ssl_module = 'pyopenssl'
"""The name of a registered SSL adaptation module to use with the builtin
WSGI server. Builtin options are 'builtin' (to use the SSL library built
into recent versions of Python) and 'pyopenssl' (to use the PyOpenSSL
project, which you must install separately). You may also register your
own classes in the wsgiserver.ssl_adapters dict."""
"""The name of a registered SSL adaptation module to use with the
builtin WSGI server. Builtin options are 'builtin' (to use the SSL
library built into recent versions of Python) and 'pyopenssl' (to
use the PyOpenSSL project, which you must install separately). You
may also register your own classes in the wsgiserver.ssl_adapters
dict."""
statistics = False
"""Turns statistics-gathering on or off for aware HTTP servers."""
@@ -157,6 +174,7 @@ class Server(ServerAdapter):
if self.socket_host is None and self.socket_port is None:
return None
return (self.socket_host, self.socket_port)
def _set_bind_addr(self, value):
if value is None:
self.socket_file = None
@@ -174,11 +192,15 @@ class Server(ServerAdapter):
raise ValueError("bind_addr must be a (host, port) tuple "
"(for TCP sockets) or a string (for Unix "
"domain sockets), not %r" % value)
bind_addr = property(_get_bind_addr, _set_bind_addr,
doc='A (host, port) tuple for TCP sockets or a str for Unix domain sockets.')
bind_addr = property(
_get_bind_addr,
_set_bind_addr,
doc='A (host, port) tuple for TCP sockets or '
'a str for Unix domain sockets.')
def base(self):
"""Return the base (scheme://host[:port] or sock file) for this server."""
"""Return the base (scheme://host[:port] or sock file) for this server.
"""
if self.socket_file:
return self.socket_file
@@ -202,4 +224,3 @@ class Server(ServerAdapter):
host += ":%s" % port
return "%s://%s" % (scheme, host)

6
lib/cherrypy/_cpthreadinglocal.py
View File

@@ -137,6 +137,7 @@ affects what we see:
# Threading import is at end
class _localbase(object):
__slots__ = '_local__key', '_local__args', '_local__lock'
@@ -158,6 +159,7 @@ class _localbase(object):
return self
def _patch(self):
key = object.__getattribute__(self, '_local__key')
d = currentThread().__dict__.get(key)
@@ -175,6 +177,7 @@ def _patch(self):
else:
object.__setattr__(self, '__dict__', d)
class local(_localbase):
def __getattribute__(self, name):
@@ -204,7 +207,6 @@ class local(_localbase):
finally:
lock.release()
def __del__():
threading_enumerate = enumerate
__getattribute__ = object.__getattribute__
@@ -231,7 +233,7 @@ class local(_localbase):
try:
del __dict__[key]
except KeyError:
pass # didn't have anything in this thread
pass # didn't have anything in this thread
return __del__
__del__ = __del__()

51
lib/cherrypy/_cptools.py
View File

@@ -43,10 +43,14 @@ def _getargs(func):
return co.co_varnames[:co.co_argcount]
_attr_error = ("CherryPy Tools cannot be turned on directly. Instead, turn them "
"on via config, or use them as decorators on your page handlers.")
_attr_error = (
"CherryPy Tools cannot be turned on directly. Instead, turn them "
"on via config, or use them as decorators on your page handlers."
)
class Tool(object):
"""A registered function for use with CherryPy request-processing hooks.
help(tool.callable) should give you more information about this Tool.
@@ -64,6 +68,7 @@ class Tool(object):
def _get_on(self):
raise AttributeError(_attr_error)
def _set_on(self, value):
raise AttributeError(_attr_error)
on = property(_get_on, _set_on)
@@ -117,6 +122,7 @@ class Tool(object):
raise TypeError("The %r Tool does not accept positional "
"arguments; you must use keyword arguments."
% self._name)
def tool_decorator(f):
if not hasattr(f, "_cp_config"):
f._cp_config = {}
@@ -142,6 +148,7 @@ class Tool(object):
class HandlerTool(Tool):
"""Tool which is called 'before main', that may skip normal handlers.
If the tool successfully handles the request (by setting response.body),
@@ -191,6 +198,7 @@ class HandlerTool(Tool):
class HandlerWrapperTool(Tool):
"""Tool which wraps request.handler in a provided wrapper function.
The 'newhandler' arg must be a handler wrapper function that takes a
@@ -209,20 +217,23 @@ class HandlerWrapperTool(Tool):
cherrypy.tools.jinja = HandlerWrapperTool(interpolator)
"""
def __init__(self, newhandler, point='before_handler', name=None, priority=50):
def __init__(self, newhandler, point='before_handler', name=None,
priority=50):
self.newhandler = newhandler
self._point = point
self._name = name
self._priority = priority
def callable(self, debug=False):
def callable(self, *args, **kwargs):
innerfunc = cherrypy.serving.request.handler
def wrap(*args, **kwargs):
return self.newhandler(innerfunc, *args, **kwargs)
cherrypy.serving.request.handler = wrap
class ErrorTool(Tool):
"""Tool which is used to replace the default request.error_response."""
def __init__(self, callable, name=None):
@@ -249,6 +260,7 @@ from cherrypy.lib import auth_basic, auth_digest
class SessionTool(Tool):
"""Session Tool for CherryPy.
sessions.locking
@@ -258,7 +270,8 @@ class SessionTool(Tool):
When 'early', the session will be locked before reading the request
body. This is off by default for safety reasons; for example,
a large upload would block the session, denying an AJAX
progress meter (see http://www.cherrypy.org/ticket/630).
progress meter
(`issue <https://bitbucket.org/cherrypy/cherrypy/issue/630>`_).
When 'explicit' (or any other value), you need to call
cherrypy.session.acquire_lock() yourself before using
@@ -314,9 +327,8 @@ class SessionTool(Tool):
_sessions.set_response_cookie(**conf)
class XMLRPCController(object):
"""A Controller (page handler collection) for XML-RPC.
To use it, have your controllers subclass this base class (it will
@@ -364,7 +376,7 @@ class XMLRPCController(object):
body = subhandler(*(vpath + rpcparams), **params)
else:
# http://www.cherrypy.org/ticket/533
# https://bitbucket.org/cherrypy/cherrypy/issue/533
# if a method is not found, an xmlrpclib.Fault should be returned
# raising an exception here will do that; see
# cherrypy.lib.xmlrpcutil.on_error
@@ -387,6 +399,7 @@ class SessionAuthTool(HandlerTool):
class CachingTool(Tool):
"""Caching Tool for CherryPy."""
def _wrapper(self, **kwargs):
@@ -397,7 +410,7 @@ class CachingTool(Tool):
if request.cacheable:
# Note the devious technique here of adding hooks on the fly
request.hooks.attach('before_finalize', _caching.tee_output,
priority = 90)
priority=90)
_wrapper.priority = 20
def _setup(self):
@@ -409,8 +422,8 @@ class CachingTool(Tool):
priority=p, **conf)
class Toolbox(object):
"""A collection of Tools.
This object also functions as a config namespace handler for itself.
@@ -431,6 +444,7 @@ class Toolbox(object):
def __enter__(self):
"""Populate request.toolmaps from tools specified in config."""
cherrypy.serving.request.toolmaps[self.namespace] = map = {}
def populate(k, v):
toolname, arg = k.split(".", 1)
bucket = map.setdefault(toolname, {})
@@ -459,6 +473,7 @@ class DeprecatedTool(Tool):
def __call__(self, *args, **kwargs):
warnings.warn(self.warnmsg)
def tool_decorator(f):
return f
return tool_decorator
@@ -487,12 +502,16 @@ _d.sessions = SessionTool()
_d.xmlrpc = ErrorTool(_xmlrpc.on_error)
_d.caching = CachingTool('before_handler', _caching.get, 'caching')
_d.expires = Tool('before_finalize', _caching.expires)
_d.tidy = DeprecatedTool('before_finalize',
"The tidy tool has been removed from the standard distribution of CherryPy. "
"The most recent version can be found at http://tools.cherrypy.org/browser.")
_d.nsgmls = DeprecatedTool('before_finalize',
"The nsgmls tool has been removed from the standard distribution of CherryPy. "
"The most recent version can be found at http://tools.cherrypy.org/browser.")
_d.tidy = DeprecatedTool(
'before_finalize',
"The tidy tool has been removed from the standard distribution of "
"CherryPy. The most recent version can be found at "
"http://tools.cherrypy.org/browser.")
_d.nsgmls = DeprecatedTool(
'before_finalize',
"The nsgmls tool has been removed from the standard distribution of "
"CherryPy. The most recent version can be found at "
"http://tools.cherrypy.org/browser.")
_d.ignore_headers = Tool('before_request_body', cptools.ignore_headers)
_d.referer = Tool('before_request_body', cptools.referer)
_d.basic_auth = Tool('on_start_resource', auth.basic_auth)

37
lib/cherrypy/_cptree.py
View File

@@ -1,7 +1,6 @@
"""CherryPy Application and Tree objects."""
import os
import sys
import cherrypy
from cherrypy._cpcompat import ntou, py3k
@@ -10,6 +9,7 @@ from cherrypy.lib import httputil
class Application(object):
"""A CherryPy Application.
Servers and gateways should not instantiate Request objects directly.
@@ -62,10 +62,10 @@ class Application(object):
return "%s.%s(%r, %r)" % (self.__module__, self.__class__.__name__,
self.root, self.script_name)
script_name_doc = """The URI "mount point" for this app. A mount point is that portion of
the URI which is constant for all URIs that are serviced by this
application; it does not include scheme, host, or proxy ("virtual host")
portions of the URI.
script_name_doc = """The URI "mount point" for this app. A mount point
is that portion of the URI which is constant for all URIs that are
serviced by this application; it does not include scheme, host, or proxy
("virtual host") portions of the URI.
For example, if script_name is "/my/cool/app", then the URL
"http://www.example.com/my/cool/app/page1" might be handled by a
@@ -77,11 +77,15 @@ class Application(object):
If script_name is explicitly set to None, then the script_name will be
provided for each call from request.wsgi_environ['SCRIPT_NAME'].
"""
def _get_script_name(self):
if self._script_name is None:
# None signals that the script name should be pulled from WSGI environ.
return cherrypy.serving.request.wsgi_environ['SCRIPT_NAME'].rstrip("/")
return self._script_name
if self._script_name is not None:
return self._script_name
# A `_script_name` with a value of None signals that the script name
# should be pulled from WSGI environ.
return cherrypy.serving.request.wsgi_environ['SCRIPT_NAME'].rstrip("/")
def _set_script_name(self, value):
if value:
value = value.rstrip("/")
@@ -148,6 +152,7 @@ class Application(object):
class Tree(object):
"""A registry of CherryPy applications, mounted at diverse points.
An instance of this class may also be used as a WSGI callable
@@ -201,8 +206,9 @@ class Tree(object):
if isinstance(root, Application):
app = root
if script_name != "" and script_name != app.script_name:
raise ValueError("Cannot specify a different script name and "
"pass an Application instance to cherrypy.mount")
raise ValueError(
"Cannot specify a different script name and pass an "
"Application instance to cherrypy.mount")
script_name = app.script_name
else:
app = Application(root, script_name)
@@ -273,7 +279,8 @@ class Tree(object):
# Python 2/WSGI u.0: all strings MUST be of type unicode
enc = environ[ntou('wsgi.url_encoding')]
environ[ntou('SCRIPT_NAME')] = sn.decode(enc)
environ[ntou('PATH_INFO')] = path[len(sn.rstrip("/")):].decode(enc)
environ[ntou('PATH_INFO')] = path[
len(sn.rstrip("/")):].decode(enc)
else:
# Python 2/WSGI 1.x: all strings MUST be of type str
environ['SCRIPT_NAME'] = sn
@@ -285,6 +292,8 @@ class Tree(object):
environ['PATH_INFO'] = path[len(sn.rstrip("/")):]
else:
# Python 3/WSGI 1.x: all strings MUST be ISO-8859-1 str
environ['SCRIPT_NAME'] = sn.encode('utf-8').decode('ISO-8859-1')
environ['PATH_INFO'] = path[len(sn.rstrip("/")):].encode('utf-8').decode('ISO-8859-1')
environ['SCRIPT_NAME'] = sn.encode(
'utf-8').decode('ISO-8859-1')
environ['PATH_INFO'] = path[
len(sn.rstrip("/")):].encode('utf-8').decode('ISO-8859-1')
return app(environ, start_response)

70
lib/cherrypy/_cpwsgi.py
View File

@@ -13,10 +13,11 @@ import cherrypy as _cherrypy
from cherrypy._cpcompat import BytesIO, bytestr, ntob, ntou, py3k, unicodestr
from cherrypy import _cperror
from cherrypy.lib import httputil
from cherrypy.lib import is_closable_iterator
def downgrade_wsgi_ux_to_1x(environ):
"""Return a new environ dict for WSGI 1.x from the given WSGI u.x environ."""
"""Return a new environ dict for WSGI 1.x from the given WSGI u.x environ.
"""
env1x = {}
url_encoding = environ[ntou('wsgi.url_encoding')]
@@ -31,6 +32,7 @@ def downgrade_wsgi_ux_to_1x(environ):
class VirtualHost(object):
"""Select a different WSGI application based on the Host header.
This can be useful when running multiple sites within one CP server.
@@ -82,6 +84,7 @@ class VirtualHost(object):
class InternalRedirector(object):
"""WSGI middleware that handles raised cherrypy.InternalRedirect."""
def __init__(self, nextapp, recursive=False):
@@ -107,7 +110,8 @@ class InternalRedirector(object):
redirections.append(old_uri)
if not self.recursive:
# Check to see if the new URI has been redirected to already
# Check to see if the new URI has been redirected to
# already
new_uri = sn + ir.path
if ir.query_string:
new_uri += "?" + ir.query_string
@@ -126,6 +130,7 @@ class InternalRedirector(object):
class ExceptionTrapper(object):
"""WSGI middleware that traps exceptions."""
def __init__(self, nextapp, throws=(KeyboardInterrupt, SystemExit)):
@@ -133,7 +138,12 @@ class ExceptionTrapper(object):
self.throws = throws
def __call__(self, environ, start_response):
return _TrappedResponse(self.nextapp, environ, start_response, self.throws)
return _TrappedResponse(
self.nextapp,
environ,
start_response,
self.throws
)
class _TrappedResponse(object):
@@ -146,7 +156,8 @@ class _TrappedResponse(object):
self.start_response = start_response
self.throws = throws
self.started_response = False
self.response = self.trap(self.nextapp, self.environ, self.start_response)
self.response = self.trap(
self.nextapp, self.environ, self.start_response)
self.iter_response = iter(self.response)
def __iter__(self):
@@ -210,6 +221,7 @@ class _TrappedResponse(object):
class AppResponse(object):
"""WSGI response iterable for CherryPy applications."""
def __init__(self, environ, start_response, cpapp):
@@ -230,16 +242,20 @@ class AppResponse(object):
outheaders = []
for k, v in r.header_list:
if not isinstance(k, bytestr):
raise TypeError("response.header_list key %r is not a byte string." % k)
raise TypeError(
"response.header_list key %r is not a byte string." %
k)
if not isinstance(v, bytestr):
raise TypeError("response.header_list value %r is not a byte string." % v)
raise TypeError(
"response.header_list value %r is not a byte string." %
v)
outheaders.append((k, v))
if py3k:
# According to PEP 3333, when using Python 3, the response status
# and headers must be bytes masquerading as unicode; that is, they
# must be of type "str" but are restricted to code points in the
# "latin-1" set.
# According to PEP 3333, when using Python 3, the response
# status and headers must be bytes masquerading as unicode;
# that is, they must be of type "str" but are restricted to
# code points in the "latin-1" set.
outstatus = outstatus.decode('ISO-8859-1')
outheaders = [(k.decode('ISO-8859-1'), v.decode('ISO-8859-1'))
for k, v in outheaders]
@@ -262,14 +278,26 @@ class AppResponse(object):
def close(self):
"""Close and de-reference the current request and response. (Core)"""
streaming = _cherrypy.serving.response.stream
self.cpapp.release_serving()
# We avoid the expense of examining the iterator to see if it's
# closable unless we are streaming the response, as that's the
# only situation where we are going to have an iterator which
# may not have been exhausted yet.
if streaming and is_closable_iterator(self.iter_response):
iter_close = self.iter_response.close
try:
iter_close()
except Exception:
_cherrypy.log(traceback=True, severity=40)
def run(self):
"""Create a Request object using environ."""
env = self.environ.get
local = httputil.Host('', int(env('SERVER_PORT', 80)),
env('SERVER_NAME', ''))
env('SERVER_NAME', ''))
remote = httputil.Host(env('REMOTE_ADDR', ''),
int(env('REMOTE_PORT', -1) or -1),
env('REMOTE_HOST', ''))
@@ -293,16 +321,17 @@ class AppResponse(object):
qs = self.environ.get('QUERY_STRING', '')
if py3k:
# This isn't perfect; if the given PATH_INFO is in the wrong encoding,
# it may fail to match the appropriate config section URI. But meh.
# This isn't perfect; if the given PATH_INFO is in the
# wrong encoding, it may fail to match the appropriate config
# section URI. But meh.
old_enc = self.environ.get('wsgi.url_encoding', 'ISO-8859-1')
new_enc = self.cpapp.find_config(self.environ.get('PATH_INFO', ''),
"request.uri_encoding", 'utf-8')
if new_enc.lower() != old_enc.lower():
# Even though the path and qs are unicode, the WSGI server is
# required by PEP 3333 to coerce them to ISO-8859-1 masquerading
# as unicode. So we have to encode back to bytes and then decode
# again using the "correct" encoding.
# Even though the path and qs are unicode, the WSGI server
# is required by PEP 3333 to coerce them to ISO-8859-1
# masquerading as unicode. So we have to encode back to
# bytes and then decode again using the "correct" encoding.
try:
u_path = path.encode(old_enc).decode(new_enc)
u_qs = qs.encode(old_enc).decode(new_enc)
@@ -339,6 +368,7 @@ class AppResponse(object):
class CPWSGIApp(object):
"""A WSGI application object for a CherryPy Application."""
pipeline = [('ExceptionTrapper', ExceptionTrapper),
@@ -361,7 +391,8 @@ class CPWSGIApp(object):
named WSGI callable (from the pipeline) as keyword arguments."""
response_class = AppResponse
"""The class to instantiate and return as the next app in the WSGI chain."""
"""The class to instantiate and return as the next app in the WSGI chain.
"""
def __init__(self, cpapp, pipeline=None):
self.cpapp = cpapp
@@ -405,4 +436,3 @@ class CPWSGIApp(object):
name, arg = k.split(".", 1)
bucket = self.config.setdefault(name, {})
bucket[arg] = v

23
lib/cherrypy/_cpwsgi_server.py
View File

@@ -8,6 +8,7 @@ from cherrypy import wsgiserver
class CPWSGIServer(wsgiserver.CherryPyWSGIServer):
"""Wrapper for wsgiserver.CherryPyWSGIServer.
wsgiserver has been designed to not reference CherryPy in any way,
@@ -18,8 +19,12 @@ class CPWSGIServer(wsgiserver.CherryPyWSGIServer):
def __init__(self, server_adapter=cherrypy.server):
self.server_adapter = server_adapter
self.max_request_header_size = self.server_adapter.max_request_header_size or 0
self.max_request_body_size = self.server_adapter.max_request_body_size or 0
self.max_request_header_size = (
self.server_adapter.max_request_header_size or 0
)
self.max_request_body_size = (
self.server_adapter.max_request_body_size or 0
)
server_name = (self.server_adapter.socket_host or
self.server_adapter.socket_file or
@@ -30,10 +35,12 @@ class CPWSGIServer(wsgiserver.CherryPyWSGIServer):
s.__init__(self, server_adapter.bind_addr, cherrypy.tree,
self.server_adapter.thread_pool,
server_name,
max = self.server_adapter.thread_pool_max,
request_queue_size = self.server_adapter.socket_queue_size,
timeout = self.server_adapter.socket_timeout,
shutdown_timeout = self.server_adapter.shutdown_timeout,
max=self.server_adapter.thread_pool_max,
request_queue_size=self.server_adapter.socket_queue_size,
timeout=self.server_adapter.socket_timeout,
shutdown_timeout=self.server_adapter.shutdown_timeout,
accepted_queue_size=self.server_adapter.accepted_queue_size,
accepted_queue_timeout=self.server_adapter.accepted_queue_timeout,
)
self.protocol = self.server_adapter.protocol_version
self.nodelay = self.server_adapter.nodelay
@@ -56,8 +63,8 @@ class CPWSGIServer(wsgiserver.CherryPyWSGIServer):
self.server_adapter.ssl_private_key,
self.server_adapter.ssl_certificate_chain)
self.stats['Enabled'] = getattr(self.server_adapter, 'statistics', False)
self.stats['Enabled'] = getattr(
self.server_adapter, 'statistics', False)
def error_log(self, msg="", level=20, traceback=False):
cherrypy.engine.log(msg, level, traceback)

29
lib/cherrypy/cherryd Executable file → Normal file
View File

@@ -7,6 +7,7 @@ import cherrypy
from cherrypy.process import plugins, servers
from cherrypy import Application
def start(configfiles=None, daemonize=False, environment=None,
fastcgi=False, scgi=False, pidfile=None, imports=None,
cgi=False):
@@ -14,7 +15,7 @@ def start(configfiles=None, daemonize=False, environment=None,
sys.path = [''] + sys.path
for i in imports or []:
exec("import %s" % i)
for c in configfiles or []:
cherrypy.config.update(c)
# If there's only one app mounted, merge config into it.
@@ -22,26 +23,26 @@ def start(configfiles=None, daemonize=False, environment=None,
for app in cherrypy.tree.apps.values():
if isinstance(app, Application):
app.merge(c)
engine = cherrypy.engine
if environment is not None:
cherrypy.config.update({'environment': environment})
# Only daemonize if asked to.
if daemonize:
# Don't print anything to stdout/sterr.
cherrypy.config.update({'log.screen': False})
plugins.Daemonizer(engine).subscribe()
if pidfile:
plugins.PIDFile(engine, pidfile).subscribe()
if hasattr(engine, "signal_handler"):
engine.signal_handler.subscribe()
if hasattr(engine, "console_control_handler"):
engine.console_control_handler.subscribe()
if (fastcgi and (scgi or cgi)) or (scgi and cgi):
cherrypy.log.error("You may only specify one of the cgi, fastcgi, and "
"scgi options.", 'ENGINE')
@@ -51,7 +52,7 @@ def start(configfiles=None, daemonize=False, environment=None,
cherrypy.config.update({'engine.autoreload_on': False})
# Turn off the default HTTP server (which is subscribed by default).
cherrypy.server.unsubscribe()
addr = cherrypy.server.bind_addr
if fastcgi:
f = servers.FlupFCGIServer(application=cherrypy.tree,
@@ -64,7 +65,7 @@ def start(configfiles=None, daemonize=False, environment=None,
bindAddress=addr)
s = servers.ServerAdapter(engine, httpserver=f, bind_addr=addr)
s.subscribe()
# Always start the engine; this will start all other services
try:
engine.start()
@@ -77,7 +78,7 @@ def start(configfiles=None, daemonize=False, environment=None,
if __name__ == '__main__':
from optparse import OptionParser
p = OptionParser()
p.add_option('-c', '--config', action="append", dest='config',
help="specify config file(s)")
@@ -86,7 +87,8 @@ if __name__ == '__main__':
p.add_option('-e', '--environment', dest='environment', default=None,
help="apply the given config environment")
p.add_option('-f', action="store_true", dest='fastcgi',
help="start a fastcgi server instead of the default HTTP server")
help="start a fastcgi server instead of the default HTTP "
"server")
p.add_option('-s', action="store_true", dest='scgi',
help="start a scgi server instead of the default HTTP server")
p.add_option('-x', action="store_true", dest='cgi',
@@ -98,12 +100,11 @@ if __name__ == '__main__':
p.add_option('-P', '--Path', action="append", dest='Path',
help="add the given paths to sys.path")
options, args = p.parse_args()
if options.Path:
for p in options.Path:
sys.path.insert(0, p)
start(options.config, options.daemonize,
options.environment, options.fastcgi, options.scgi,
options.pidfile, options.imports, options.cgi)

40
lib/cherrypy/lib/__init__.py
View File

@@ -3,7 +3,45 @@
# Deprecated in CherryPy 3.2 -- remove in CherryPy 3.3
from cherrypy.lib.reprconf import unrepr, modules, attributes
def is_iterator(obj):
'''Returns a boolean indicating if the object provided implements
the iterator protocol (i.e. like a generator). This will return
false for objects which iterable, but not iterators themselves.'''
from types import GeneratorType
if isinstance(obj, GeneratorType):
return True
elif not hasattr(obj, '__iter__'):
return False
else:
# Types which implement the protocol must return themselves when
# invoking 'iter' upon them.
return iter(obj) is obj
def is_closable_iterator(obj):
# Not an iterator.
if not is_iterator(obj):
return False
# A generator - the easiest thing to deal with.
import inspect
if inspect.isgenerator(obj):
return True
# A custom iterator. Look for a close method...
if not (hasattr(obj, 'close') and callable(obj.close)):
return False
# ... which doesn't require any arguments.
try:
inspect.getcallargs(obj.close)
except TypeError:
return False
else:
return True
class file_generator(object):
"""Yield the given input (a file object) in chunks (default 64k). (Core)"""
def __init__(self, input, chunkSize=65536):
@@ -23,6 +61,7 @@ class file_generator(object):
raise StopIteration()
next = __next__
def file_generator_limited(fileobj, count, chunk_size=65536):
"""Yield the given file object in chunks, stopping after `count`
bytes has been emitted. Default chunk size is 64kB. (Core)
@@ -36,6 +75,7 @@ def file_generator_limited(fileobj, count, chunk_size=65536):
remaining -= chunklen
yield chunk
def set_vary_header(response, header_name):
"Add a Vary header to a response"
varies = response.headers.get("Vary", "")

28
lib/cherrypy/lib/auth.py
View File

@@ -3,7 +3,8 @@ from cherrypy.lib import httpauth
def check_auth(users, encrypt=None, realm=None):
"""If an authorization header contains credentials, return True, else False."""
"""If an authorization header contains credentials, return True or False.
"""
request = cherrypy.serving.request
if 'authorization' in request.headers:
# make sure the provided credentials are correctly set
@@ -17,10 +18,11 @@ def check_auth(users, encrypt=None, realm=None):
if hasattr(users, '__call__'):
try:
# backward compatibility
users = users() # expect it to return a dictionary
users = users() # expect it to return a dictionary
if not isinstance(users, dict):
raise ValueError("Authentication users must be a dictionary")
raise ValueError(
"Authentication users must be a dictionary")
# fetch the user password
password = users.get(ah["username"], None)
@@ -44,6 +46,7 @@ def check_auth(users, encrypt=None, realm=None):
request.login = False
return False
def basic_auth(realm, users, encrypt=None, debug=False):
"""If auth fails, raise 401 with a basic authentication header.
@@ -51,7 +54,8 @@ def basic_auth(realm, users, encrypt=None, debug=False):
A string containing the authentication realm.
users
A dict of the form: {username: password} or a callable returning a dict.
A dict of the form: {username: password} or a callable returning
a dict.
encrypt
callable used to encrypt the password returned from the user-agent.
@@ -64,9 +68,12 @@ def basic_auth(realm, users, encrypt=None, debug=False):
return
# inform the user-agent this path is protected
cherrypy.serving.response.headers['www-authenticate'] = httpauth.basicAuth(realm)
cherrypy.serving.response.headers[
'www-authenticate'] = httpauth.basicAuth(realm)
raise cherrypy.HTTPError(
401, "You are not authorized to access that resource")
raise cherrypy.HTTPError(401, "You are not authorized to access that resource")
def digest_auth(realm, users, debug=False):
"""If auth fails, raise 401 with a digest authentication header.
@@ -74,7 +81,8 @@ def digest_auth(realm, users, debug=False):
realm
A string containing the authentication realm.
users
A dict of the form: {username: password} or a callable returning a dict.
A dict of the form: {username: password} or a callable returning
a dict.
"""
if check_auth(users, realm=realm):
if debug:
@@ -82,6 +90,8 @@ def digest_auth(realm, users, debug=False):
return
# inform the user-agent this path is protected
cherrypy.serving.response.headers['www-authenticate'] = httpauth.digestAuth(realm)
cherrypy.serving.response.headers[
'www-authenticate'] = httpauth.digestAuth(realm)
raise cherrypy.HTTPError(401, "You are not authorized to access that resource")
raise cherrypy.HTTPError(
401, "You are not authorized to access that resource")

15
lib/cherrypy/lib/auth_basic.py
View File

@@ -3,7 +3,8 @@
# vim:ts=4:sw=4:expandtab:fileencoding=utf-8
__doc__ = """This module provides a CherryPy 3.x tool which implements
the server-side of HTTP Basic Access Authentication, as described in :rfc:`2617`.
the server-side of HTTP Basic Access Authentication, as described in
:rfc:`2617`.
Example usage, using the built-in checkpassword_dict function which uses a dict
as the credentials store::
@@ -77,11 +78,13 @@ def basic_auth(realm, checkpassword, debug=False):
if debug:
cherrypy.log('Auth succeeded', 'TOOLS.AUTH_BASIC')
request.login = username
return # successful authentication
except (ValueError, binascii.Error): # split() error, base64.decodestring() error
return # successful authentication
# split() error, base64.decodestring() error
except (ValueError, binascii.Error):
raise cherrypy.HTTPError(400, 'Bad Request')
# Respond with 401 status and a WWW-Authenticate header
cherrypy.serving.response.headers['www-authenticate'] = 'Basic realm="%s"' % realm
raise cherrypy.HTTPError(401, "You are not authorized to access that resource")
cherrypy.serving.response.headers[
'www-authenticate'] = 'Basic realm="%s"' % realm
raise cherrypy.HTTPError(
401, "You are not authorized to access that resource")

141
lib/cherrypy/lib/auth_digest.py
View File

@@ -41,6 +41,8 @@ def TRACE(msg):
# Three helper functions for users of the tool, providing three variants
# of get_ha1() functions for three different kinds of credential stores.
def get_ha1_dict_plain(user_password_dict):
"""Returns a get_ha1 function which obtains a plaintext password from a
dictionary of the form: {username : password}.
@@ -57,6 +59,7 @@ def get_ha1_dict_plain(user_password_dict):
return get_ha1
def get_ha1_dict(user_ha1_dict):
"""Returns a get_ha1 function which obtains a HA1 password hash from a
dictionary of the form: {username : HA1}.
@@ -67,10 +70,11 @@ def get_ha1_dict(user_ha1_dict):
argument to digest_auth().
"""
def get_ha1(realm, username):
return user_ha1_dict.get(user)
return user_ha1_dict.get(username)
return get_ha1
def get_ha1_file_htdigest(filename):
"""Returns a get_ha1 function which obtains a HA1 password hash from a
flat file with lines of the same format as that produced by the Apache
@@ -99,8 +103,9 @@ def get_ha1_file_htdigest(filename):
def synthesize_nonce(s, key, timestamp=None):
"""Synthesize a nonce value which resists spoofing and can be checked for staleness.
Returns a string suitable as the value for 'nonce' in the www-authenticate header.
"""Synthesize a nonce value which resists spoofing and can be checked
for staleness. Returns a string suitable as the value for 'nonce' in
the www-authenticate header.
s
A string related to the resource, such as the hostname of the server.
@@ -125,6 +130,7 @@ def H(s):
class HttpDigestAuthorization (object):
"""Class to parse a Digest Authorization header and perform re-calculation
of the digest.
"""
@@ -135,7 +141,7 @@ class HttpDigestAuthorization (object):
def __init__(self, auth_header, http_method, debug=False):
self.http_method = http_method
self.debug = debug
scheme, params = auth_header.split(" ", 1)
scheme, params = auth_header.split(" ", 1)
self.scheme = scheme.lower()
if self.scheme != 'digest':
raise ValueError('Authorization scheme is not "Digest"')
@@ -151,84 +157,95 @@ class HttpDigestAuthorization (object):
self.nonce = paramsd.get('nonce')
self.uri = paramsd.get('uri')
self.method = paramsd.get('method')
self.response = paramsd.get('response') # the response digest
self.algorithm = paramsd.get('algorithm', 'MD5')
self.response = paramsd.get('response') # the response digest
self.algorithm = paramsd.get('algorithm', 'MD5').upper()
self.cnonce = paramsd.get('cnonce')
self.opaque = paramsd.get('opaque')
self.qop = paramsd.get('qop') # qop
self.nc = paramsd.get('nc') # nonce count
self.qop = paramsd.get('qop') # qop
self.nc = paramsd.get('nc') # nonce count
# perform some correctness checks
if self.algorithm not in valid_algorithms:
raise ValueError(self.errmsg("Unsupported value for algorithm: '%s'" % self.algorithm))
raise ValueError(
self.errmsg("Unsupported value for algorithm: '%s'" %
self.algorithm))
has_reqd = self.username and \
self.realm and \
self.nonce and \
self.uri and \
self.response
has_reqd = (
self.username and
self.realm and
self.nonce and
self.uri and
self.response
)
if not has_reqd:
raise ValueError(self.errmsg("Not all required parameters are present."))
raise ValueError(
self.errmsg("Not all required parameters are present."))
if self.qop:
if self.qop not in valid_qops:
raise ValueError(self.errmsg("Unsupported value for qop: '%s'" % self.qop))
raise ValueError(
self.errmsg("Unsupported value for qop: '%s'" % self.qop))
if not (self.cnonce and self.nc):
raise ValueError(self.errmsg("If qop is sent then cnonce and nc MUST be present"))
raise ValueError(
self.errmsg("If qop is sent then "
"cnonce and nc MUST be present"))
else:
if self.cnonce or self.nc:
raise ValueError(self.errmsg("If qop is not sent, neither cnonce nor nc can be present"))
raise ValueError(
self.errmsg("If qop is not sent, "
"neither cnonce nor nc can be present"))
def __str__(self):
return 'authorization : %s' % self.auth_header
def validate_nonce(self, s, key):
"""Validate the nonce.
Returns True if nonce was generated by synthesize_nonce() and the timestamp
is not spoofed, else returns False.
Returns True if nonce was generated by synthesize_nonce() and the
timestamp is not spoofed, else returns False.
s
A string related to the resource, such as the hostname of the server.
A string related to the resource, such as the hostname of
the server.
key
A secret string known only to the server.
Both s and key must be the same values which were used to synthesize the nonce
we are trying to validate.
Both s and key must be the same values which were used to synthesize
the nonce we are trying to validate.
"""
try:
timestamp, hashpart = self.nonce.split(':', 1)
s_timestamp, s_hashpart = synthesize_nonce(s, key, timestamp).split(':', 1)
s_timestamp, s_hashpart = synthesize_nonce(
s, key, timestamp).split(':', 1)
is_valid = s_hashpart == hashpart
if self.debug:
TRACE('validate_nonce: %s' % is_valid)
return is_valid
except ValueError: # split() error
except ValueError: # split() error
pass
return False
def is_nonce_stale(self, max_age_seconds=600):
"""Returns True if a validated nonce is stale. The nonce contains a
timestamp in plaintext and also a secure hash of the timestamp. You should
first validate the nonce to ensure the plaintext timestamp is not spoofed.
timestamp in plaintext and also a secure hash of the timestamp.
You should first validate the nonce to ensure the plaintext
timestamp is not spoofed.
"""
try:
timestamp, hashpart = self.nonce.split(':', 1)
if int(timestamp) + max_age_seconds > int(time.time()):
return False
except ValueError: # int() error
except ValueError: # int() error
pass
if self.debug:
TRACE("nonce is stale")
return True
def HA2(self, entity_body=''):
"""Returns the H(A2) string. See :rfc:`2617` section 3.2.2.3."""
# RFC 2617 3.2.2.3
# If the "qop" directive's value is "auth" or is unspecified, then A2 is:
# If the "qop" directive's value is "auth" or is unspecified,
# then A2 is:
# A2 = method ":" digest-uri-value
#
# If the "qop" value is "auth-int", then A2 is:
@@ -238,11 +255,11 @@ class HttpDigestAuthorization (object):
elif self.qop == "auth-int":
a2 = "%s:%s:%s" % (self.http_method, self.uri, H(entity_body))
else:
# in theory, this should never happen, since I validate qop in __init__()
# in theory, this should never happen, since I validate qop in
# __init__()
raise ValueError(self.errmsg("Unrecognized value for qop!"))
return H(a2)
def request_digest(self, ha1, entity_body=''):
"""Calculates the Request-Digest. See :rfc:`2617` section 3.2.2.1.
@@ -253,22 +270,24 @@ class HttpDigestAuthorization (object):
If 'qop' is set to 'auth-int', then A2 includes a hash
of the "entity body". The entity body is the part of the
message which follows the HTTP headers. See :rfc:`2617` section
4.3. This refers to the entity the user agent sent in the request which
has the Authorization header. Typically GET requests don't have an entity,
and POST requests do.
4.3. This refers to the entity the user agent sent in the
request which has the Authorization header. Typically GET
requests don't have an entity, and POST requests do.
"""
ha2 = self.HA2(entity_body)
# Request-Digest -- RFC 2617 3.2.2.1
if self.qop:
req = "%s:%s:%s:%s:%s" % (self.nonce, self.nc, self.cnonce, self.qop, ha2)
req = "%s:%s:%s:%s:%s" % (
self.nonce, self.nc, self.cnonce, self.qop, ha2)
else:
req = "%s:%s" % (self.nonce, ha2)
# RFC 2617 3.2.2.2
#
# If the "algorithm" directive's value is "MD5" or is unspecified, then A1 is:
# A1 = unq(username-value) ":" unq(realm-value) ":" passwd
# If the "algorithm" directive's value is "MD5" or is unspecified,
# then A1 is:
# A1 = unq(username-value) ":" unq(realm-value) ":" passwd
#
# If the "algorithm" directive's value is "MD5-sess", then A1 is
# calculated only once - on the first request by the client following
@@ -282,8 +301,8 @@ class HttpDigestAuthorization (object):
return digest
def www_authenticate(realm, key, algorithm='MD5', nonce=None, qop=qop_auth, stale=False):
def www_authenticate(realm, key, algorithm='MD5', nonce=None, qop=qop_auth,
stale=False):
"""Constructs a WWW-Authenticate header for Digest authentication."""
if qop not in valid_qops:
raise ValueError("Unsupported value for qop: '%s'" % qop)
@@ -293,7 +312,7 @@ def www_authenticate(realm, key, algorithm='MD5', nonce=None, qop=qop_auth, stal
if nonce is None:
nonce = synthesize_nonce(realm, key)
s = 'Digest realm="%s", nonce="%s", algorithm="%s", qop="%s"' % (
realm, nonce, algorithm, qop)
realm, nonce, algorithm, qop)
if stale:
s += ', stale="true"'
return s
@@ -303,11 +322,11 @@ def digest_auth(realm, get_ha1, key, debug=False):
"""A CherryPy tool which hooks at before_handler to perform
HTTP Digest Access Authentication, as specified in :rfc:`2617`.
If the request has an 'authorization' header with a 'Digest' scheme, this
tool authenticates the credentials supplied in that header. If
the request has no 'authorization' header, or if it does but the scheme is
not "Digest", or if authentication fails, the tool sends a 401 response with
a 'WWW-Authenticate' Digest header.
If the request has an 'authorization' header with a 'Digest' scheme,
this tool authenticates the credentials supplied in that header.
If the request has no 'authorization' header, or if it does but the
scheme is not "Digest", or if authentication fails, the tool sends
a 401 response with a 'WWW-Authenticate' Digest header.
realm
A string containing the authentication realm.
@@ -322,7 +341,8 @@ def digest_auth(realm, get_ha1, key, debug=False):
None.
key
A secret string known only to the server, used in the synthesis of nonces.
A secret string known only to the server, used in the synthesis
of nonces.
"""
request = cherrypy.serving.request
@@ -331,9 +351,11 @@ def digest_auth(realm, get_ha1, key, debug=False):
nonce_is_stale = False
if auth_header is not None:
try:
auth = HttpDigestAuthorization(auth_header, request.method, debug=debug)
auth = HttpDigestAuthorization(
auth_header, request.method, debug=debug)
except ValueError:
raise cherrypy.HTTPError(400, "The Authorization header could not be parsed.")
raise cherrypy.HTTPError(
400, "The Authorization header could not be parsed.")
if debug:
TRACE(str(auth))
@@ -341,19 +363,22 @@ def digest_auth(realm, get_ha1, key, debug=False):
if auth.validate_nonce(realm, key):
ha1 = get_ha1(realm, auth.username)
if ha1 is not None:
# note that for request.body to be available we need to hook in at
# before_handler, not on_start_resource like 3.1.x digest_auth does.
# note that for request.body to be available we need to
# hook in at before_handler, not on_start_resource like
# 3.1.x digest_auth does.
digest = auth.request_digest(ha1, entity_body=request.body)
if digest == auth.response: # authenticated
if digest == auth.response: # authenticated
if debug:
TRACE("digest matches auth.response")
# Now check if nonce is stale.
# The choice of ten minutes' lifetime for nonce is somewhat arbitrary
# The choice of ten minutes' lifetime for nonce is somewhat
# arbitrary
nonce_is_stale = auth.is_nonce_stale(max_age_seconds=600)
if not nonce_is_stale:
request.login = auth.username
if debug:
TRACE("authentication of %s successful" % auth.username)
TRACE("authentication of %s successful" %
auth.username)
return
# Respond with 401 status and a WWW-Authenticate header
@@ -361,5 +386,5 @@ def digest_auth(realm, get_ha1, key, debug=False):
if debug:
TRACE(header)
cherrypy.serving.response.headers['WWW-Authenticate'] = header
raise cherrypy.HTTPError(401, "You are not authorized to access that resource")
raise cherrypy.HTTPError(
401, "You are not authorized to access that resource")

43
lib/cherrypy/lib/caching.py
View File

@@ -1,7 +1,7 @@
"""
CherryPy implements a simple caching system as a pluggable Tool. This tool tries
to be an (in-process) HTTP/1.1-compliant cache. It's not quite there yet, but
it's probably good enough for most sites.
CherryPy implements a simple caching system as a pluggable Tool. This tool
tries to be an (in-process) HTTP/1.1-compliant cache. It's not quite there
yet, but it's probably good enough for most sites.
In general, GET responses are cached (along with selecting headers) and, if
another request arrives for the same resource, the caching Tool will return 304
@@ -9,8 +9,8 @@ Not Modified if possible, or serve the cached response otherwise. It also sets
request.cached to True if serving a cached representation, and sets
request.cacheable to False (so it doesn't get cached again).
If POST, PUT, or DELETE requests are made for a cached resource, they invalidate
(delete) any cached response.
If POST, PUT, or DELETE requests are made for a cached resource, they
invalidate (delete) any cached response.
Usage
=====
@@ -39,10 +39,11 @@ import time
import cherrypy
from cherrypy.lib import cptools, httputil
from cherrypy._cpcompat import copyitems, ntob, set_daemon, sorted
from cherrypy._cpcompat import copyitems, ntob, set_daemon, sorted, Event
class Cache(object):
"""Base class for Cache implementations."""
def get(self):
@@ -62,11 +63,9 @@ class Cache(object):
raise NotImplemented
# ------------------------------- Memory Cache ------------------------------- #
# ------------------------------ Memory Cache ------------------------------- #
class AntiStampedeCache(dict):
"""A storage system for cached items which reduces stampede collisions."""
def wait(self, key, timeout=5, debug=False):
@@ -81,7 +80,7 @@ class AntiStampedeCache(dict):
If timeout is None, no waiting is performed nor sentinels used.
"""
value = self.get(key)
if isinstance(value, threading._Event):
if isinstance(value, Event):
if timeout is None:
# Ignore the other thread and recalc it ourselves.
if debug:
@@ -90,7 +89,8 @@ class AntiStampedeCache(dict):
# Wait until it's done or times out.
if debug:
cherrypy.log('Waiting up to %s seconds' % timeout, 'TOOLS.CACHING')
cherrypy.log('Waiting up to %s seconds' %
timeout, 'TOOLS.CACHING')
value.wait(timeout)
if value.result is not None:
# The other thread finished its calculation. Use it.
@@ -120,7 +120,7 @@ class AntiStampedeCache(dict):
"""Set the cached value for the given key."""
existing = self.get(key)
dict.__setitem__(self, key, value)
if isinstance(existing, threading._Event):
if isinstance(existing, Event):
# Set Event.result so other threads waiting on it have
# immediate access without needing to poll the cache again.
existing.result = value
@@ -128,6 +128,7 @@ class AntiStampedeCache(dict):
class MemoryCache(Cache):
"""An in-memory cache for varying response content.
Each key in self.store is a URI, and each value is an AntiStampedeCache.
@@ -152,7 +153,8 @@ class MemoryCache(Cache):
"""The maximum size of the entire cache in bytes; defaults to 10 MB."""
delay = 600
"""Seconds until the cached content expires; defaults to 600 (10 minutes)."""
"""Seconds until the cached content expires; defaults to 600 (10 minutes).
"""
antistampede_timeout = 5
"""Seconds to wait for other threads to release a cache lock."""
@@ -325,13 +327,15 @@ def get(invalid_methods=("POST", "PUT", "DELETE"), debug=False, **kwargs):
directive = atoms.pop(0)
if directive == 'max-age':
if len(atoms) != 1 or not atoms[0].isdigit():
raise cherrypy.HTTPError(400, "Invalid Cache-Control header")
raise cherrypy.HTTPError(
400, "Invalid Cache-Control header")
max_age = int(atoms[0])
break
elif directive == 'no-cache':
if debug:
cherrypy.log('Ignoring cache due to Cache-Control: no-cache',
'TOOLS.CACHING')
cherrypy.log(
'Ignoring cache due to Cache-Control: no-cache',
'TOOLS.CACHING')
request.cached = False
request.cacheable = True
return False
@@ -348,7 +352,8 @@ def get(invalid_methods=("POST", "PUT", "DELETE"), debug=False, **kwargs):
request.cacheable = True
return False
# Copy the response headers. See http://www.cherrypy.org/ticket/721.
# Copy the response headers. See
# https://bitbucket.org/cherrypy/cherrypy/issue/721.
response.headers = rh = httputil.HeaderMap()
for k in h:
dict.__setitem__(rh, k, dict.__getitem__(h, k))
@@ -387,7 +392,7 @@ def tee_output():
def tee(body):
"""Tee response.body into a list."""
if ('no-cache' in response.headers.values('Pragma') or
'no-store' in response.headers.values('Cache-Control')):
'no-store' in response.headers.values('Cache-Control')):
for chunk in body:
yield chunk
return

50
lib/cherrypy/lib/covercp.py
View File

@@ -24,13 +24,15 @@ import re
import sys
import cgi
from cherrypy._cpcompat import quote_plus
import os, os.path
import os
import os.path
localFile = os.path.join(os.path.dirname(__file__), "coverage.cache")
the_coverage = None
try:
from coverage import coverage
the_coverage = coverage(data_file=localFile)
def start():
the_coverage.start()
except ImportError:
@@ -39,7 +41,9 @@ except ImportError:
the_coverage = None
import warnings
warnings.warn("No code coverage will be performed; coverage.py could not be imported.")
warnings.warn(
"No code coverage will be performed; "
"coverage.py could not be imported.")
def start():
pass
@@ -118,10 +122,13 @@ TEMPLATE_FORM = """
<div id="options">
<form action='menu' method=GET>
<input type='hidden' name='base' value='%(base)s' />
Show percentages <input type='checkbox' %(showpct)s name='showpct' value='checked' /><br />
Hide files over <input type='text' id='pct' name='pct' value='%(pct)s' size='3' />%%<br />
Show percentages
<input type='checkbox' %(showpct)s name='showpct' value='checked' /><br />
Hide files over
<input type='text' id='pct' name='pct' value='%(pct)s' size='3' />%%<br />
Exclude files matching<br />
<input type='text' id='exclude' name='exclude' value='%(exclude)s' size='20' />
<input type='text' id='exclude' name='exclude'
value='%(exclude)s' size='20' />
<br />
<input type='submit' value='Change view' id="submit"/>
@@ -173,7 +180,10 @@ TEMPLATE_LOC_EXCLUDED = """<tr class="excluded">
<td>%s</td>
</tr>\n"""
TEMPLATE_ITEM = "%s%s<a class='file' href='report?name=%s' target='main'>%s</a>\n"
TEMPLATE_ITEM = (
"%s%s<a class='file' href='report?name=%s' target='main'>%s</a>\n"
)
def _percent(statements, missing):
s = len(statements)
@@ -182,6 +192,7 @@ def _percent(statements, missing):
return int(round(100.0 * e / s))
return 0
def _show_branch(root, base, path, pct=0, showpct=False, exclude="",
coverage=the_coverage):
@@ -194,10 +205,16 @@ def _show_branch(root, base, path, pct=0, showpct=False, exclude="",
if newpath.lower().startswith(base):
relpath = newpath[len(base):]
yield "| " * relpath.count(os.sep)
yield "<a class='directory' href='menu?base=%s&exclude=%s'>%s</a>\n" % \
(newpath, quote_plus(exclude), name)
yield (
"<a class='directory' "
"href='menu?base=%s&exclude=%s'>%s</a>\n" %
(newpath, quote_plus(exclude), name)
)
for chunk in _show_branch(root[name], base, newpath, pct, showpct, exclude, coverage=coverage):
for chunk in _show_branch(
root[name], base, newpath, pct, showpct,
exclude, coverage=coverage
):
yield chunk
# Now list the files
@@ -217,7 +234,7 @@ def _show_branch(root, base, path, pct=0, showpct=False, exclude="",
pass
else:
pc = _percent(statements, missing)
pc_str = ("%3d%% " % pc).replace(' ','&nbsp;')
pc_str = ("%3d%% " % pc).replace(' ', '&nbsp;')
if pc < float(pct) or pc == -1:
pc_str = "<span class='fail'>%s</span>" % pc_str
else:
@@ -226,10 +243,12 @@ def _show_branch(root, base, path, pct=0, showpct=False, exclude="",
yield TEMPLATE_ITEM % ("| " * (relpath.count(os.sep) + 1),
pc_str, newpath, name)
def _skip_file(path, exclude):
if exclude:
return bool(re.search(exclude, path))
def _graft(path, tree):
d = tree
@@ -249,6 +268,7 @@ def _graft(path, tree):
if node:
d = d.setdefault(node, {})
def get_tree(base, exclude, coverage=the_coverage):
"""Return covered module names as a nested dict."""
tree = {}
@@ -258,6 +278,7 @@ def get_tree(base, exclude, coverage=the_coverage):
_graft(path, tree)
return tree
class CoverStats(object):
def __init__(self, coverage, root=None):
@@ -301,7 +322,8 @@ class CoverStats(object):
yield "<p>No modules covered.</p>"
else:
for chunk in _show_branch(tree, base, "/", pct,
showpct=='checked', exclude, coverage=self.coverage):
showpct == 'checked', exclude,
coverage=self.coverage):
yield chunk
yield "</div>"
@@ -331,7 +353,8 @@ class CoverStats(object):
yield template % (lineno, cgi.escape(line))
def report(self, name):
filename, statements, excluded, missing, _ = self.coverage.analysis2(name)
filename, statements, excluded, missing, _ = self.coverage.analysis2(
name)
pc = _percent(statements, missing)
yield TEMPLATE_COVERAGE % dict(name=os.path.basename(name),
fullpath=name,
@@ -350,7 +373,7 @@ def serve(path=localFile, port=8080, root=None):
if coverage is None:
raise ImportError("The coverage module could not be imported.")
from coverage import coverage
cov = coverage(data_file = path)
cov = coverage(data_file=path)
cov.load()
import cherrypy
@@ -362,4 +385,3 @@ def serve(path=localFile, port=8080, root=None):
if __name__ == "__main__":
serve(*tuple(sys.argv[1:]))

181
lib/cherrypy/lib/cpstats.py
View File

@@ -21,33 +21,37 @@ to collect stats to import a third-party module. Therefore, we choose to
re-use the `logging` module by adding a `statistics` object to it.
That `logging.statistics` object is a nested dict. It is not a custom class,
because that would 1) require libraries and applications to import a third-
party module in order to participate, 2) inhibit innovation in extrapolation
approaches and in reporting tools, and 3) be slow. There are, however, some
specifications regarding the structure of the dict.
because that would:
{
+----"SQLAlchemy": {
| "Inserts": 4389745,
| "Inserts per Second":
| lambda s: s["Inserts"] / (time() - s["Start"]),
| C +---"Table Statistics": {
| o | "widgets": {-----------+
N | l | "Rows": 1.3M, | Record
a | l | "Inserts": 400, |
m | e | },---------------------+
e | c | "froobles": {
s | t | "Rows": 7845,
p | i | "Inserts": 0,
a | o | },
c | n +---},
e | "Slow Queries":
| [{"Query": "SELECT * FROM widgets;",
| "Processing Time": 47.840923343,
| },
| ],
+----},
}
1. require libraries and applications to import a third-party module in
order to participate
2. inhibit innovation in extrapolation approaches and in reporting tools, and
3. be slow.
There are, however, some specifications regarding the structure of the dict.::
{
+----"SQLAlchemy": {
| "Inserts": 4389745,
| "Inserts per Second":
| lambda s: s["Inserts"] / (time() - s["Start"]),
| C +---"Table Statistics": {
| o | "widgets": {-----------+
N | l | "Rows": 1.3M, | Record
a | l | "Inserts": 400, |
m | e | },---------------------+
e | c | "froobles": {
s | t | "Rows": 7845,
p | i | "Inserts": 0,
a | o | },
c | n +---},
e | "Slow Queries":
| [{"Query": "SELECT * FROM widgets;",
| "Processing Time": 47.840923343,
| },
| ],
+----},
}
The `logging.statistics` dict has four levels. The topmost level is nothing
more than a set of names to introduce modularity, usually along the lines of
@@ -65,13 +69,13 @@ Each namespace, then, is a dict of named statistical values, such as
good on a report: spaces and capitalization are just fine.
In addition to scalars, values in a namespace MAY be a (third-layer)
dict, or a list, called a "collection". For example, the CherryPy StatsTool
keeps track of what each request is doing (or has most recently done)
in a 'Requests' collection, where each key is a thread ID; each
dict, or a list, called a "collection". For example, the CherryPy
:class:`StatsTool` keeps track of what each request is doing (or has most
recently done) in a 'Requests' collection, where each key is a thread ID; each
value in the subdict MUST be a fourth dict (whew!) of statistical data about
each thread. We call each subdict in the collection a "record". Similarly,
the StatsTool also keeps a list of slow queries, where each record contains
data about each slow query, in order.
the :class:`StatsTool` also keeps a list of slow queries, where each record
contains data about each slow query, in order.
Values in a namespace or record may also be functions, which brings us to:
@@ -86,17 +90,17 @@ scalar values you already have on hand.
When it comes time to report on the gathered data, however, we usually have
much more freedom in what we can calculate. Therefore, whenever reporting
tools (like the provided StatsPage CherryPy class) fetch the contents of
`logging.statistics` for reporting, they first call `extrapolate_statistics`
(passing the whole `statistics` dict as the only argument). This makes a
deep copy of the statistics dict so that the reporting tool can both iterate
over it and even change it without harming the original. But it also expands
any functions in the dict by calling them. For example, you might have a
'Current Time' entry in the namespace with the value "lambda scope: time.time()".
The "scope" parameter is the current namespace dict (or record, if we're
currently expanding one of those instead), allowing you access to existing
static entries. If you're truly evil, you can even modify more than one entry
at a time.
tools (like the provided :class:`StatsPage` CherryPy class) fetch the contents
of `logging.statistics` for reporting, they first call
`extrapolate_statistics` (passing the whole `statistics` dict as the only
argument). This makes a deep copy of the statistics dict so that the
reporting tool can both iterate over it and even change it without harming
the original. But it also expands any functions in the dict by calling them.
For example, you might have a 'Current Time' entry in the namespace with the
value "lambda scope: time.time()". The "scope" parameter is the current
namespace dict (or record, if we're currently expanding one of those
instead), allowing you access to existing static entries. If you're truly
evil, you can even modify more than one entry at a time.
However, don't try to calculate an entry and then use its value in further
extrapolations; the order in which the functions are called is not guaranteed.
@@ -108,19 +112,20 @@ After the whole thing has been extrapolated, it's time for:
Reporting
---------
The StatsPage class grabs the `logging.statistics` dict, extrapolates it all,
and then transforms it to HTML for easy viewing. Each namespace gets its own
header and attribute table, plus an extra table for each collection. This is
NOT part of the statistics specification; other tools can format how they like.
The :class:`StatsPage` class grabs the `logging.statistics` dict, extrapolates
it all, and then transforms it to HTML for easy viewing. Each namespace gets
its own header and attribute table, plus an extra table for each collection.
This is NOT part of the statistics specification; other tools can format how
they like.
You can control which columns are output and how they are formatted by updating
StatsPage.formatting, which is a dict that mirrors the keys and nesting of
`logging.statistics`. The difference is that, instead of data values, it has
formatting values. Use None for a given key to indicate to the StatsPage that a
given column should not be output. Use a string with formatting (such as '%.3f')
to interpolate the value(s), or use a callable (such as lambda v: v.isoformat())
for more advanced formatting. Any entry which is not mentioned in the formatting
dict is output unchanged.
given column should not be output. Use a string with formatting
(such as '%.3f') to interpolate the value(s), or use a callable (such as
lambda v: v.isoformat()) for more advanced formatting. Any entry which is not
mentioned in the formatting dict is output unchanged.
Monitoring
----------
@@ -145,12 +150,12 @@ entries to False or True, if present.
Usage
=====
To collect statistics on CherryPy applications:
To collect statistics on CherryPy applications::
from cherrypy.lib import cpstats
appconfig['/']['tools.cpstats.on'] = True
To collect statistics on your own code:
To collect statistics on your own code::
import logging
# Initialize the repository
@@ -172,20 +177,22 @@ To collect statistics on your own code:
if mystats.get('Enabled', False):
mystats['Important Events'] += 1
To report statistics:
To report statistics::
root.cpstats = cpstats.StatsPage()
To format statistics reports:
To format statistics reports::
See 'Reporting', above.
"""
# -------------------------------- Statistics -------------------------------- #
# ------------------------------- Statistics -------------------------------- #
import logging
if not hasattr(logging, 'statistics'): logging.statistics = {}
if not hasattr(logging, 'statistics'):
logging.statistics = {}
def extrapolate_statistics(scope):
"""Return an extrapolated copy of the given scope."""
@@ -201,7 +208,7 @@ def extrapolate_statistics(scope):
return c
# --------------------- CherryPy Applications Statistics --------------------- #
# -------------------- CherryPy Applications Statistics --------------------- #
import threading
import time
@@ -211,12 +218,20 @@ import cherrypy
appstats = logging.statistics.setdefault('CherryPy Applications', {})
appstats.update({
'Enabled': True,
'Bytes Read/Request': lambda s: (s['Total Requests'] and
(s['Total Bytes Read'] / float(s['Total Requests'])) or 0.0),
'Bytes Read/Request': lambda s: (
s['Total Requests'] and
(s['Total Bytes Read'] / float(s['Total Requests'])) or
0.0
),
'Bytes Read/Second': lambda s: s['Total Bytes Read'] / s['Uptime'](s),
'Bytes Written/Request': lambda s: (s['Total Requests'] and
(s['Total Bytes Written'] / float(s['Total Requests'])) or 0.0),
'Bytes Written/Second': lambda s: s['Total Bytes Written'] / s['Uptime'](s),
'Bytes Written/Request': lambda s: (
s['Total Requests'] and
(s['Total Bytes Written'] / float(s['Total Requests'])) or
0.0
),
'Bytes Written/Second': lambda s: (
s['Total Bytes Written'] / s['Uptime'](s)
),
'Current Time': lambda s: time.time(),
'Current Requests': 0,
'Requests/Second': lambda s: float(s['Total Requests']) / s['Uptime'](s),
@@ -228,12 +243,13 @@ appstats.update({
'Total Time': 0,
'Uptime': lambda s: time.time() - s['Start Time'],
'Requests': {},
})
})
proc_time = lambda s: time.time() - s['Start Time']
class ByteCountWrapper(object):
"""Wraps a file-like object, counting the number of bytes read."""
def __init__(self, rfile):
@@ -279,6 +295,7 @@ average_uriset_time = lambda s: s['Count'] and (s['Sum'] / s['Count']) or 0
class StatsTool(cherrypy.Tool):
"""Record various information about the current request."""
def __init__(self):
@@ -315,10 +332,11 @@ class StatsTool(cherrypy.Tool):
'Request-Line': request.request_line,
'Response Status': None,
'Start Time': time.time(),
}
}
def record_stop(self, uriset=None, slow_queries=1.0, slow_queries_count=100,
debug=False, **kwargs):
def record_stop(
self, uriset=None, slow_queries=1.0, slow_queries_count=100,
debug=False, **kwargs):
"""Record the end of a request."""
resp = cherrypy.serving.response
w = appstats['Requests'][threading._get_ident()]
@@ -334,7 +352,8 @@ class StatsTool(cherrypy.Tool):
w['Bytes Written'] = cl
appstats['Total Bytes Written'] += cl
w['Response Status'] = getattr(resp, 'output_status', None) or resp.status
w['Response Status'] = getattr(
resp, 'output_status', None) or resp.status
w['End Time'] = time.time()
p = w['End Time'] - w['Start Time']
@@ -388,6 +407,7 @@ missing = object()
locale_date = lambda v: time.strftime('%c', time.gmtime(v))
iso_format = lambda v: time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime(v))
def pause_resume(ns):
def _pause_resume(enabled):
pause_disabled = ''
@@ -427,20 +447,20 @@ class StatsPage(object):
'End Time': None,
'Processing Time': '%.3f',
'Start Time': iso_format,
},
},
'URI Set Tracking': {
'Avg': '%.3f',
'Max': '%.3f',
'Min': '%.3f',
'Sum': '%.3f',
},
},
'Requests': {
'Bytes Read': '%s',
'Bytes Written': '%s',
'End Time': None,
'Processing Time': '%.3f',
'Start Time': None,
},
},
},
'CherryPy WSGIServer': {
'Enabled': pause_resume('CherryPy WSGIServer'),
@@ -449,7 +469,6 @@ class StatsPage(object):
},
}
def index(self):
# Transform the raw data into pretty output for HTML
yield """
@@ -500,18 +519,25 @@ table.stats2 th {
""" % title
for i, (key, value) in enumerate(scalars):
colnum = i % 3
if colnum == 0: yield """
if colnum == 0:
yield """
<tr>"""
yield """
<th>%(key)s</th><td id='%(title)s-%(key)s'>%(value)s</td>""" % vars()
if colnum == 2: yield """
yield (
"""
<th>%(key)s</th><td id='%(title)s-%(key)s'>%(value)s</td>""" %
vars()
)
if colnum == 2:
yield """
</tr>"""
if colnum == 0: yield """
if colnum == 0:
yield """
<th></th><td></td>
<th></th><td></td>
</tr>"""
elif colnum == 1: yield """
elif colnum == 1:
yield """
<th></th><td></td>
</tr>"""
yield """
@@ -659,4 +685,3 @@ table.stats2 th {
resume.exposed = True
resume.cp_config = {'tools.allow.on': True,
'tools.allow.methods': ['POST']}

85
lib/cherrypy/lib/cptools.py
View File

@@ -4,8 +4,9 @@ import logging
import re
import cherrypy
from cherrypy._cpcompat import basestring, ntob, md5, set
from cherrypy._cpcompat import basestring, md5, set, unicodestr
from cherrypy.lib import httputil as _httputil
from cherrypy.lib import is_iterator
# Conditional HTTP request support #
@@ -79,13 +80,15 @@ def validate_etags(autotags=False, debug=False):
'TOOLS.ETAGS')
if conditions == ["*"] or etag in conditions:
if debug:
cherrypy.log('request.method: %s' % request.method, 'TOOLS.ETAGS')
cherrypy.log('request.method: %s' %
request.method, 'TOOLS.ETAGS')
if request.method in ("GET", "HEAD"):
raise cherrypy.HTTPRedirect([], 304)
else:
raise cherrypy.HTTPError(412, "If-None-Match failed: ETag %r "
"matched %r" % (etag, conditions))
def validate_since():
"""Validate the current Last-Modified against If-Modified-Since headers.
@@ -207,8 +210,8 @@ def proxy(base=None, local='X-Forwarded-Host', remote='X-Forwarded-For',
cherrypy.log('Testing remote %r:%r' % (remote, xff), 'TOOLS.PROXY')
if xff:
if remote == 'X-Forwarded-For':
# See http://bob.pythonmac.org/archives/2005/09/23/apache-x-forwarded-for-caveat/
xff = xff.split(',')[-1].strip()
#Bug #1268
xff = xff.split(',')[0].strip()
request.remote.ip = xff
@@ -277,6 +280,7 @@ def referer(pattern, accept=True, accept_missing=False, error=403,
class SessionAuth(object):
"""Assert that the user is logged in."""
session_key = "username"
@@ -298,17 +302,20 @@ class SessionAuth(object):
def on_check(self, username):
pass
def login_screen(self, from_page='..', username='', error_msg='', **kwargs):
return ntob("""<html><body>
def login_screen(self, from_page='..', username='', error_msg='',
**kwargs):
return (unicodestr("""<html><body>
Message: %(error_msg)s
<form method="post" action="do_login">
Login: <input type="text" name="username" value="%(username)s" size="10" /><br />
Password: <input type="password" name="password" size="10" /><br />
<input type="hidden" name="from_page" value="%(from_page)s" /><br />
Login: <input type="text" name="username" value="%(username)s" size="10" />
<br />
Password: <input type="password" name="password" size="10" />
<br />
<input type="hidden" name="from_page" value="%(from_page)s" />
<br />
<input type="submit" />
</form>
</body></html>""" % {'from_page': from_page, 'username': username,
'error_msg': error_msg}, "utf-8")
</body></html>""") % vars()).encode("utf-8")
def do_login(self, username, password, from_page='..', **kwargs):
"""Login. May raise redirect, or return True if request handled."""
@@ -338,7 +345,8 @@ Message: %(error_msg)s
raise cherrypy.HTTPRedirect(from_page)
def do_check(self):
"""Assert username. May raise redirect, or return True if request handled."""
"""Assert username. Raise redirect, or return True if request handled.
"""
sess = cherrypy.session
request = cherrypy.serving.request
response = cherrypy.serving.response
@@ -346,51 +354,51 @@ Message: %(error_msg)s
username = sess.get(self.session_key)
if not username:
sess[self.session_key] = username = self.anonymous()
if self.debug:
cherrypy.log('No session[username], trying anonymous', 'TOOLS.SESSAUTH')
self._debug_message('No session[username], trying anonymous')
if not username:
url = cherrypy.url(qs=request.query_string)
if self.debug:
cherrypy.log('No username, routing to login_screen with '
'from_page %r' % url, 'TOOLS.SESSAUTH')
self._debug_message(
'No username, routing to login_screen with from_page %(url)r',
locals(),
)
response.body = self.login_screen(url)
if "Content-Length" in response.headers:
# Delete Content-Length header so finalize() recalcs it.
del response.headers["Content-Length"]
return True
if self.debug:
cherrypy.log('Setting request.login to %r' % username, 'TOOLS.SESSAUTH')
self._debug_message('Setting request.login to %(username)r', locals())
request.login = username
self.on_check(username)
def _debug_message(self, template, context={}):
if not self.debug:
return
cherrypy.log(template % context, 'TOOLS.SESSAUTH')
def run(self):
request = cherrypy.serving.request
response = cherrypy.serving.response
path = request.path_info
if path.endswith('login_screen'):
if self.debug:
cherrypy.log('routing %r to login_screen' % path, 'TOOLS.SESSAUTH')
return self.login_screen(**request.params)
self._debug_message('routing %(path)r to login_screen', locals())
response.body = self.login_screen()
return True
elif path.endswith('do_login'):
if request.method != 'POST':
response.headers['Allow'] = "POST"
if self.debug:
cherrypy.log('do_login requires POST', 'TOOLS.SESSAUTH')
self._debug_message('do_login requires POST')
raise cherrypy.HTTPError(405)
if self.debug:
cherrypy.log('routing %r to do_login' % path, 'TOOLS.SESSAUTH')
self._debug_message('routing %(path)r to do_login', locals())
return self.do_login(**request.params)
elif path.endswith('do_logout'):
if request.method != 'POST':
response.headers['Allow'] = "POST"
raise cherrypy.HTTPError(405)
if self.debug:
cherrypy.log('routing %r to do_logout' % path, 'TOOLS.SESSAUTH')
self._debug_message('routing %(path)r to do_logout', locals())
return self.do_logout(**request.params)
else:
if self.debug:
cherrypy.log('No special path, running do_check', 'TOOLS.SESSAUTH')
self._debug_message('No special path, running do_check')
return self.do_check()
@@ -412,11 +420,13 @@ def log_traceback(severity=logging.ERROR, debug=False):
"""Write the last error's traceback to the cherrypy error log."""
cherrypy.log("", "HTTP", severity=severity, traceback=True)
def log_request_headers(debug=False):
"""Write request headers to the cherrypy error log."""
h = [" %s: %s" % (k, v) for k, v in cherrypy.serving.request.header_list]
cherrypy.log('\nRequest Headers:\n' + '\n'.join(h), "HTTP")
def log_hooks(debug=False):
"""Write request.hooks to the cherrypy error log."""
request = cherrypy.serving.request
@@ -438,6 +448,7 @@ def log_hooks(debug=False):
cherrypy.log('\nRequest Hooks for ' + cherrypy.url() +
':\n' + '\n'.join(msg), "HTTP")
def redirect(url='', internal=True, debug=False):
"""Raise InternalRedirect or HTTPRedirect to the given url."""
if debug:
@@ -449,6 +460,7 @@ def redirect(url='', internal=True, debug=False):
else:
raise cherrypy.HTTPRedirect(url)
def trailing_slash(missing=True, extra=False, status=None, debug=False):
"""Redirect if path_info has (missing|extra) trailing slash."""
request = cherrypy.serving.request
@@ -470,17 +482,17 @@ def trailing_slash(missing=True, extra=False, status=None, debug=False):
new_url = cherrypy.url(pi[:-1], request.query_string)
raise cherrypy.HTTPRedirect(new_url, status=status or 301)
def flatten(debug=False):
"""Wrap response.body in a generator that recursively iterates over body.
This allows cherrypy.response.body to consist of 'nested generators';
that is, a set of generators that yield generators.
"""
import types
def flattener(input):
numchunks = 0
for x in input:
if not isinstance(x, types.GeneratorType):
if not is_iterator(x):
numchunks += 1
yield x
else:
@@ -593,7 +605,8 @@ class MonitoredHeaderMap(_httputil.HeaderMap):
def autovary(ignore=None, debug=False):
"""Auto-populate the Vary response header based on request.header access."""
"""Auto-populate the Vary response header based on request.header access.
"""
request = cherrypy.serving.request
req_h = request.headers
@@ -606,12 +619,12 @@ def autovary(ignore=None, debug=False):
resp_h = cherrypy.serving.response.headers
v = set([e.value for e in resp_h.elements('Vary')])
if debug:
cherrypy.log('Accessed headers: %s' % request.headers.accessed_headers,
'TOOLS.AUTOVARY')
cherrypy.log(
'Accessed headers: %s' % request.headers.accessed_headers,
'TOOLS.AUTOVARY')
v = v.union(request.headers.accessed_headers)
v = v.difference(ignore)
v = list(v)
v.sort()
resp_h['Vary'] = ', '.join(v)
request.hooks.attach('before_finalize', set_response_header, 95)

101
lib/cherrypy/lib/encoding.py
View File

@@ -4,6 +4,7 @@ import time
import cherrypy
from cherrypy._cpcompat import basestring, BytesIO, ntob, set, unicodestr
from cherrypy.lib import file_generator
from cherrypy.lib import is_closable_iterator
from cherrypy.lib import set_vary_header
@@ -14,13 +15,13 @@ def decode(encoding=None, default_encoding='utf-8'):
encoding
If not None, restricts the set of charsets attempted while decoding
a request entity to the given set (even if a different charset is given in
the Content-Type request header).
a request entity to the given set (even if a different charset is
given in the Content-Type request header).
default_encoding
Only in effect if the 'encoding' argument is not given.
If given, the set of charsets attempted while decoding a request entity is
*extended* with the given value(s).
If given, the set of charsets attempted while decoding a request
entity is *extended* with the given value(s).
"""
body = cherrypy.request.body
@@ -33,6 +34,31 @@ def decode(encoding=None, default_encoding='utf-8'):
default_encoding = [default_encoding]
body.attempt_charsets = body.attempt_charsets + default_encoding
class UTF8StreamEncoder:
def __init__(self, iterator):
self._iterator = iterator
def __iter__(self):
return self
def next(self):
return self.__next__()
def __next__(self):
res = next(self._iterator)
if isinstance(res, unicodestr):
res = res.encode('utf-8')
return res
def close(self):
if is_closable_iterator(self._iterator):
self._iterator.close()
def __getattr__(self, attr):
if attr.startswith('__'):
raise AttributeError(self, attr)
return getattr(self._iterator, attr)
class ResponseEncoder:
@@ -80,25 +106,24 @@ class ResponseEncoder:
if encoding in self.attempted_charsets:
return False
self.attempted_charsets.add(encoding)
try:
body = []
for chunk in self.body:
if isinstance(chunk, unicodestr):
body = []
for chunk in self.body:
if isinstance(chunk, unicodestr):
try:
chunk = chunk.encode(encoding, self.errors)
body.append(chunk)
self.body = body
except (LookupError, UnicodeError):
return False
else:
return True
except (LookupError, UnicodeError):
return False
body.append(chunk)
self.body = body
return True
def find_acceptable_charset(self):
request = cherrypy.serving.request
response = cherrypy.serving.response
if self.debug:
cherrypy.log('response.stream %r' % response.stream, 'TOOLS.ENCODE')
cherrypy.log('response.stream %r' %
response.stream, 'TOOLS.ENCODE')
if response.stream:
encoder = self.encode_stream
else:
@@ -127,10 +152,12 @@ class ResponseEncoder:
# If specified, force this encoding to be used, or fail.
encoding = self.encoding.lower()
if self.debug:
cherrypy.log('Specified encoding %r' % encoding, 'TOOLS.ENCODE')
cherrypy.log('Specified encoding %r' %
encoding, 'TOOLS.ENCODE')
if (not charsets) or "*" in charsets or encoding in charsets:
if self.debug:
cherrypy.log('Attempting encoding %r' % encoding, 'TOOLS.ENCODE')
cherrypy.log('Attempting encoding %r' %
encoding, 'TOOLS.ENCODE')
if encoder(encoding):
return encoding
else:
@@ -142,7 +169,8 @@ class ResponseEncoder:
if encoder(self.default_encoding):
return self.default_encoding
else:
raise cherrypy.HTTPError(500, self.failmsg % self.default_encoding)
raise cherrypy.HTTPError(500, self.failmsg %
self.default_encoding)
else:
for element in encs:
if element.qvalue > 0:
@@ -180,7 +208,8 @@ class ResponseEncoder:
msg = "Your client did not send an Accept-Charset header."
else:
msg = "Your client sent this Accept-Charset header: %s." % ac
msg += " We tried these charsets: %s." % ", ".join(self.attempted_charsets)
_charsets = ", ".join(sorted(self.attempted_charsets))
msg += " We tried these charsets: %s." % (_charsets,)
raise cherrypy.HTTPError(406, msg)
def __call__(self, *args, **kwargs):
@@ -203,39 +232,42 @@ class ResponseEncoder:
ct = response.headers.elements("Content-Type")
if self.debug:
cherrypy.log('Content-Type: %r' % [str(h) for h in ct], 'TOOLS.ENCODE')
if ct:
cherrypy.log('Content-Type: %r' % [str(h)
for h in ct], 'TOOLS.ENCODE')
if ct and self.add_charset:
ct = ct[0]
if self.text_only:
if ct.value.lower().startswith("text/"):
if self.debug:
cherrypy.log('Content-Type %s starts with "text/"' % ct,
'TOOLS.ENCODE')
cherrypy.log(
'Content-Type %s starts with "text/"' % ct,
'TOOLS.ENCODE')
do_find = True
else:
if self.debug:
cherrypy.log('Not finding because Content-Type %s does '
'not start with "text/"' % ct,
cherrypy.log('Not finding because Content-Type %s '
'does not start with "text/"' % ct,
'TOOLS.ENCODE')
do_find = False
else:
if self.debug:
cherrypy.log('Finding because not text_only', 'TOOLS.ENCODE')
cherrypy.log('Finding because not text_only',
'TOOLS.ENCODE')
do_find = True
if do_find:
# Set "charset=..." param on response Content-Type header
ct.params['charset'] = self.find_acceptable_charset()
if self.add_charset:
if self.debug:
cherrypy.log('Setting Content-Type %s' % ct,
'TOOLS.ENCODE')
response.headers["Content-Type"] = str(ct)
if self.debug:
cherrypy.log('Setting Content-Type %s' % ct,
'TOOLS.ENCODE')
response.headers["Content-Type"] = str(ct)
return self.body
# GZIP
def compress(body, compress_level):
"""Compress 'body' at the given compress_level."""
import zlib
@@ -265,6 +297,7 @@ def compress(body, compress_level):
# ISIZE: 4 bytes
yield struct.pack("<L", size & int('FFFFFFFF', 16))
def decompress(body):
import gzip
@@ -277,7 +310,8 @@ def decompress(body):
return data
def gzip(compress_level=5, mime_types=['text/html', 'text/plain'], debug=False):
def gzip(compress_level=5, mime_types=['text/html', 'text/plain'],
debug=False):
"""Try to gzip the response body if Content-Type in mime_types.
cherrypy.response.headers['Content-Type'] must be set to one of the
@@ -385,4 +419,3 @@ def gzip(compress_level=5, mime_types=['text/html', 'text/plain'], debug=False):
if debug:
cherrypy.log('No acceptable encoding found.', context='GZIP')
cherrypy.HTTPError(406, "identity, gzip").set_response()

21
lib/cherrypy/lib/gctools.py
View File

@@ -15,6 +15,7 @@ from cherrypy.process.plugins import SimplePlugin
class ReferrerTree(object):
"""An object which gathers all referrers of an object to a given depth."""
peek_length = 40
@@ -89,6 +90,7 @@ class ReferrerTree(object):
def format(self, tree):
"""Return a list of string reprs from a nested list of referrers."""
output = []
def ascend(branch, depth=1):
for parent, grandparents in branch:
output.append((" " * depth) + self._format(parent))
@@ -111,7 +113,7 @@ class RequestCounter(SimplePlugin):
self.count += 1
def after_request(self):
self.count -=1
self.count -= 1
request_counter = RequestCounter(cherrypy.engine)
request_counter.subscribe()
@@ -129,15 +131,17 @@ def get_context(obj):
class GCRoot(object):
"""A CherryPy page handler for testing reference leaks."""
classes = [(_cprequest.Request, 2, 2,
"Should be 1 in this request thread and 1 in the main thread."),
(_cprequest.Response, 2, 2,
"Should be 1 in this request thread and 1 in the main thread."),
(_cpwsgi.AppResponse, 1, 1,
"Should be 1 in this request thread only."),
]
classes = [
(_cprequest.Request, 2, 2,
"Should be 1 in this request thread and 1 in the main thread."),
(_cprequest.Response, 2, 2,
"Should be 1 in this request thread and 1 in the main thread."),
(_cpwsgi.AppResponse, 1, 1,
"Should be 1 in this request thread only."),
]
def index(self):
return "Hello, world!"
@@ -211,4 +215,3 @@ class GCRoot(object):
return "\n".join(output)
stats.exposed = True

1
lib/cherrypy/lib/http.py
View File

@@ -4,4 +4,3 @@ warnings.warn('cherrypy.lib.http has been deprecated and will be removed '
DeprecationWarning)
from cherrypy.lib.httputil import *

127
lib/cherrypy/lib/httpauth.py
View File

@@ -1,5 +1,6 @@
"""
This module defines functions to implement HTTP Digest Authentication (:rfc:`2617`).
This module defines functions to implement HTTP Digest Authentication
(:rfc:`2617`).
This has full compliance with 'Digest' and 'Basic' authentication methods. In
'Digest' it supports both MD5 and MD5-sess algorithms.
@@ -11,9 +12,10 @@ Usage:
Then use 'parseAuthorization' to retrieve the 'auth_map' used in
'checkResponse'.
To use 'checkResponse' you must have already verified the password associated
with the 'username' key in 'auth_map' dict. Then you use the 'checkResponse'
function to verify if the password matches the one sent by the client.
To use 'checkResponse' you must have already verified the password
associated with the 'username' key in 'auth_map' dict. Then you use the
'checkResponse' function to verify if the password matches the one sent
by the client.
SUPPORTED_ALGORITHM - list of supported 'Digest' algorithms
SUPPORTED_QOP - list of supported 'Digest' 'qop'.
@@ -21,7 +23,8 @@ SUPPORTED_QOP - list of supported 'Digest' 'qop'.
__version__ = 1, 0, 1
__author__ = "Tiago Cogumbreiro <cogumbreiro@users.sf.net>"
__credits__ = """
Peter van Kampen for its recipe which implement most of Digest authentication:
Peter van Kampen for its recipe which implement most of Digest
authentication:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/302378
"""
@@ -29,17 +32,17 @@ __license__ = """
Copyright (c) 2005, Tiago Cogumbreiro <cogumbreiro@users.sf.net>
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of Sylvain Hellegouarch nor the names of his contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
* Neither the name of Sylvain Hellegouarch nor the names of his
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
@@ -57,7 +60,7 @@ __all__ = ("digestAuth", "basicAuth", "doAuth", "checkResponse",
"parseAuthorization", "SUPPORTED_ALGORITHM", "md5SessionKey",
"calculateNonce", "SUPPORTED_QOP")
################################################################################
##########################################################################
import time
from cherrypy._cpcompat import base64_decode, ntob, md5
from cherrypy._cpcompat import parse_http_list, parse_keqv_list
@@ -70,16 +73,17 @@ AUTH_INT = "auth-int"
SUPPORTED_ALGORITHM = (MD5, MD5_SESS)
SUPPORTED_QOP = (AUTH, AUTH_INT)
################################################################################
##########################################################################
# doAuth
#
DIGEST_AUTH_ENCODERS = {
MD5: lambda val: md5(ntob(val)).hexdigest(),
MD5_SESS: lambda val: md5(ntob(val)).hexdigest(),
# SHA: lambda val: sha.new(ntob(val)).hexdigest (),
# SHA: lambda val: sha.new(ntob(val)).hexdigest (),
}
def calculateNonce (realm, algorithm = MD5):
def calculateNonce(realm, algorithm=MD5):
"""This is an auxaliary function that calculates 'nonce' value. It is used
to handle sessions."""
@@ -89,44 +93,47 @@ def calculateNonce (realm, algorithm = MD5):
try:
encoder = DIGEST_AUTH_ENCODERS[algorithm]
except KeyError:
raise NotImplementedError ("The chosen algorithm (%s) does not have "\
"an implementation yet" % algorithm)
raise NotImplementedError("The chosen algorithm (%s) does not have "
"an implementation yet" % algorithm)
return encoder ("%d:%s" % (time.time(), realm))
return encoder("%d:%s" % (time.time(), realm))
def digestAuth (realm, algorithm = MD5, nonce = None, qop = AUTH):
def digestAuth(realm, algorithm=MD5, nonce=None, qop=AUTH):
"""Challenges the client for a Digest authentication."""
global SUPPORTED_ALGORITHM, DIGEST_AUTH_ENCODERS, SUPPORTED_QOP
assert algorithm in SUPPORTED_ALGORITHM
assert qop in SUPPORTED_QOP
if nonce is None:
nonce = calculateNonce (realm, algorithm)
nonce = calculateNonce(realm, algorithm)
return 'Digest realm="%s", nonce="%s", algorithm="%s", qop="%s"' % (
realm, nonce, algorithm, qop
)
def basicAuth (realm):
def basicAuth(realm):
"""Challengenes the client for a Basic authentication."""
assert '"' not in realm, "Realms cannot contain the \" (quote) character."
return 'Basic realm="%s"' % realm
def doAuth (realm):
def doAuth(realm):
"""'doAuth' function returns the challenge string b giving priority over
Digest and fallback to Basic authentication when the browser doesn't
support the first one.
This should be set in the HTTP header under the key 'WWW-Authenticate'."""
return digestAuth (realm) + " " + basicAuth (realm)
return digestAuth(realm) + " " + basicAuth(realm)
################################################################################
##########################################################################
# Parse authorization parameters
#
def _parseDigestAuthorization (auth_params):
def _parseDigestAuthorization(auth_params):
# Convert the auth params to a dict
items = parse_http_list(auth_params)
params = parse_keqv_list(items)
@@ -140,8 +147,8 @@ def _parseDigestAuthorization (auth_params):
return None
# If qop is sent then cnonce and nc MUST be present
if "qop" in params and not ("cnonce" in params \
and "nc" in params):
if "qop" in params and not ("cnonce" in params
and "nc" in params):
return None
# If qop is not sent, neither cnonce nor nc can be present
@@ -152,7 +159,7 @@ def _parseDigestAuthorization (auth_params):
return params
def _parseBasicAuthorization (auth_params):
def _parseBasicAuthorization(auth_params):
username, password = base64_decode(auth_params).split(":", 1)
return {"username": username, "password": password}
@@ -161,18 +168,19 @@ AUTH_SCHEMES = {
"digest": _parseDigestAuthorization,
}
def parseAuthorization (credentials):
def parseAuthorization(credentials):
"""parseAuthorization will convert the value of the 'Authorization' key in
the HTTP header to a map itself. If the parsing fails 'None' is returned.
"""
global AUTH_SCHEMES
auth_scheme, auth_params = credentials.split(" ", 1)
auth_scheme = auth_scheme.lower ()
auth_scheme, auth_params = credentials.split(" ", 1)
auth_scheme = auth_scheme.lower()
parser = AUTH_SCHEMES[auth_scheme]
params = parser (auth_params)
params = parser(auth_params)
if params is None:
return
@@ -182,10 +190,10 @@ def parseAuthorization (credentials):
return params
################################################################################
##########################################################################
# Check provided response for a valid password
#
def md5SessionKey (params, password):
def md5SessionKey(params, password):
"""
If the "algorithm" directive's value is "MD5-sess", then A1
[the session key] is calculated only once - on the first request by the
@@ -210,10 +218,11 @@ def md5SessionKey (params, password):
params_copy[key] = params[key]
params_copy["algorithm"] = MD5_SESS
return _A1 (params_copy, password)
return _A1(params_copy, password)
def _A1(params, password):
algorithm = params.get ("algorithm", MD5)
algorithm = params.get("algorithm", MD5)
H = DIGEST_AUTH_ENCODERS[algorithm]
if algorithm == MD5:
@@ -227,7 +236,7 @@ def _A1(params, password):
# This is A1 if qop is set
# A1 = H( unq(username-value) ":" unq(realm-value) ":" passwd )
# ":" unq(nonce-value) ":" unq(cnonce-value)
h_a1 = H ("%s:%s:%s" % (params["username"], params["realm"], password))
h_a1 = H("%s:%s:%s" % (params["username"], params["realm"], password))
return "%s:%s:%s" % (h_a1, params["nonce"], params["cnonce"])
@@ -235,13 +244,13 @@ def _A2(params, method, kwargs):
# If the "qop" directive's value is "auth" or is unspecified, then A2 is:
# A2 = Method ":" digest-uri-value
qop = params.get ("qop", "auth")
qop = params.get("qop", "auth")
if qop == "auth":
return method + ":" + params["uri"]
elif qop == "auth-int":
# If the "qop" value is "auth-int", then A2 is:
# A2 = Method ":" digest-uri-value ":" H(entity-body)
entity_body = kwargs.get ("entity_body", "")
entity_body = kwargs.get("entity_body", "")
H = kwargs["H"]
return "%s:%s:%s" % (
@@ -251,20 +260,22 @@ def _A2(params, method, kwargs):
)
else:
raise NotImplementedError ("The 'qop' method is unknown: %s" % qop)
raise NotImplementedError("The 'qop' method is unknown: %s" % qop)
def _computeDigestResponse(auth_map, password, method = "GET", A1 = None,**kwargs):
def _computeDigestResponse(auth_map, password, method="GET", A1=None,
**kwargs):
"""
Generates a response respecting the algorithm defined in RFC 2617
"""
params = auth_map
algorithm = params.get ("algorithm", MD5)
algorithm = params.get("algorithm", MD5)
H = DIGEST_AUTH_ENCODERS[algorithm]
KD = lambda secret, data: H(secret + ":" + data)
qop = params.get ("qop", None)
qop = params.get("qop", None)
H_A2 = H(_A2(params, method, kwargs))
@@ -297,7 +308,8 @@ def _computeDigestResponse(auth_map, password, method = "GET", A1 = None,**kwarg
return KD(H_A1, request)
def _checkDigestResponse(auth_map, password, method = "GET", A1 = None, **kwargs):
def _checkDigestResponse(auth_map, password, method="GET", A1=None, **kwargs):
"""This function is used to verify the response given by the client when
he tries to authenticate.
Optional arguments:
@@ -312,43 +324,48 @@ def _checkDigestResponse(auth_map, password, method = "GET", A1 = None, **kwargs
if auth_map['realm'] != kwargs.get('realm', None):
return False
response = _computeDigestResponse(auth_map, password, method, A1,**kwargs)
response = _computeDigestResponse(
auth_map, password, method, A1, **kwargs)
return response == auth_map["response"]
def _checkBasicResponse (auth_map, password, method='GET', encrypt=None, **kwargs):
def _checkBasicResponse(auth_map, password, method='GET', encrypt=None,
**kwargs):
# Note that the Basic response doesn't provide the realm value so we cannot
# test it
pass_through = lambda password, username=None: password
encrypt = encrypt or pass_through
try:
return encrypt(auth_map["password"], auth_map["username"]) == password
candidate = encrypt(auth_map["password"], auth_map["username"])
except TypeError:
return encrypt(auth_map["password"]) == password
# if encrypt only takes one parameter, it's the password
candidate = encrypt(auth_map["password"])
return candidate == password
AUTH_RESPONSES = {
"basic": _checkBasicResponse,
"digest": _checkDigestResponse,
}
def checkResponse (auth_map, password, method = "GET", encrypt=None, **kwargs):
def checkResponse(auth_map, password, method="GET", encrypt=None, **kwargs):
"""'checkResponse' compares the auth_map with the password and optionally
other arguments that each implementation might need.
If the response is of type 'Basic' then the function has the following
signature::
checkBasicResponse (auth_map, password) -> bool
checkBasicResponse(auth_map, password) -> bool
If the response is of type 'Digest' then the function has the following
signature::
checkDigestResponse (auth_map, password, method = 'GET', A1 = None) -> bool
checkDigestResponse(auth_map, password, method='GET', A1=None) -> bool
The 'A1' argument is only used in MD5_SESS algorithm based responses.
Check md5SessionKey() for more info.
"""
checker = AUTH_RESPONSES[auth_map["auth_scheme"]]
return checker (auth_map, password, method=method, encrypt=encrypt, **kwargs)
return checker(auth_map, password, method=method, encrypt=encrypt,
**kwargs)

88
lib/cherrypy/lib/httputil.py
View File

@@ -8,24 +8,24 @@ to a public caning.
"""
from binascii import b2a_base64
from cherrypy._cpcompat import BaseHTTPRequestHandler, HTTPDate, ntob, ntou, reversed, sorted
from cherrypy._cpcompat import basestring, bytestr, iteritems, nativestr, unicodestr, unquote_qs
from cherrypy._cpcompat import BaseHTTPRequestHandler, HTTPDate, ntob, ntou
from cherrypy._cpcompat import basestring, bytestr, iteritems, nativestr
from cherrypy._cpcompat import reversed, sorted, unicodestr, unquote_qs
response_codes = BaseHTTPRequestHandler.responses.copy()
# From http://www.cherrypy.org/ticket/361
# From https://bitbucket.org/cherrypy/cherrypy/issue/361
response_codes[500] = ('Internal Server Error',
'The server encountered an unexpected condition '
'which prevented it from fulfilling the request.')
'The server encountered an unexpected condition '
'which prevented it from fulfilling the request.')
response_codes[503] = ('Service Unavailable',
'The server is currently unable to handle the '
'request due to a temporary overloading or '
'maintenance of the server.')
'The server is currently unable to handle the '
'request due to a temporary overloading or '
'maintenance of the server.')
import re
import urllib
def urljoin(*atoms):
"""Return the given path \*atoms, joined into a single URL.
@@ -38,6 +38,7 @@ def urljoin(*atoms):
# Special-case the final url of "", and return "/" instead.
return url or "/"
def urljoin_bytes(*atoms):
"""Return the given path *atoms, joined into a single URL.
@@ -50,10 +51,12 @@ def urljoin_bytes(*atoms):
# Special-case the final url of "", and return "/" instead.
return url or ntob("/")
def protocol_from_http(protocol_str):
"""Return a protocol tuple from the given 'HTTP/x.y' string."""
return int(protocol_str[5]), int(protocol_str[7])
def get_ranges(headervalue, content_length):
"""Return a list of (start, stop) indices from a Range header, or None.
@@ -100,12 +103,20 @@ def get_ranges(headervalue, content_length):
# See rfc quote above.
return None
# Negative subscript (last N bytes)
result.append((content_length - int(stop), content_length))
#
# RFC 2616 Section 14.35.1:
# If the entity is shorter than the specified suffix-length,
# the entire entity-body is used.
if int(stop) > content_length:
result.append((0, content_length))
else:
result.append((content_length - int(stop), content_length))
return result
class HeaderElement(object):
"""An element (with parameters) from an HTTP header's element list."""
def __init__(self, value, params=None):
@@ -122,7 +133,7 @@ class HeaderElement(object):
def __str__(self):
p = [";%s=%s" % (k, v) for k, v in iteritems(self.params)]
return "%s%s" % (self.value, "".join(p))
return str("%s%s" % (self.value, "".join(p)))
def __bytes__(self):
return ntob(self.__str__())
@@ -160,7 +171,9 @@ class HeaderElement(object):
q_separator = re.compile(r'; *q *=')
class AcceptElement(HeaderElement):
"""An element (with parameters) from an Accept* header's element list.
AcceptElement objects are comparable; the more-preferred object will be
@@ -206,14 +219,15 @@ class AcceptElement(HeaderElement):
else:
return self.qvalue < other.qvalue
RE_HEADER_SPLIT = re.compile(',(?=(?:[^"]*"[^"]*")*[^"]*$)')
def header_elements(fieldname, fieldvalue):
"""Return a sorted HeaderElement list from a comma-separated header string."""
"""Return a sorted HeaderElement list from a comma-separated header string.
"""
if not fieldvalue:
return []
result = []
for element in fieldvalue.split(","):
for element in RE_HEADER_SPLIT.split(fieldvalue):
if fieldname.startswith("Accept") or fieldname == 'TE':
hv = AcceptElement.from_str(element)
else:
@@ -222,6 +236,7 @@ def header_elements(fieldname, fieldvalue):
return list(reversed(sorted(result)))
def decode_TEXT(value):
r"""Decode :rfc:`2047` TEXT (e.g. "=?utf-8?q?f=C3=BCr?=" -> "f\xfcr")."""
try:
@@ -237,6 +252,7 @@ def decode_TEXT(value):
decodedvalue += atom
return decodedvalue
def valid_status(status):
"""Return legal HTTP status Code, Reason-phrase and Message.
@@ -332,6 +348,7 @@ def _parse_qs(qs, keep_blank_values=0, strict_parsing=0, encoding='utf-8'):
image_map_pattern = re.compile(r"[0-9]+,[0-9]+")
def parse_query_string(query_string, keep_blank_values=True, encoding='utf-8'):
"""Build a params dictionary from a query_string.
@@ -350,6 +367,7 @@ def parse_query_string(query_string, keep_blank_values=True, encoding='utf-8'):
class CaseInsensitiveDict(dict):
"""A case-insensitive dict subclass.
Each key is changed on entry to str(key).title().
@@ -372,7 +390,7 @@ class CaseInsensitiveDict(dict):
if hasattr({}, 'has_key'):
def has_key(self, key):
return dict.has_key(self, str(key).title())
return str(key).title() in self
def update(self, E):
for k in E.keys():
@@ -404,13 +422,15 @@ class CaseInsensitiveDict(dict):
# replaced with a single SP before interpretation of the TEXT value."
if nativestr == bytestr:
header_translate_table = ''.join([chr(i) for i in xrange(256)])
header_translate_deletechars = ''.join([chr(i) for i in xrange(32)]) + chr(127)
header_translate_deletechars = ''.join(
[chr(i) for i in xrange(32)]) + chr(127)
else:
header_translate_table = None
header_translate_deletechars = bytes(range(32)) + bytes([127])
class HeaderMap(CaseInsensitiveDict):
"""A dict subclass for HTTP request and response headers.
Each key is changed on entry to str(key).title(). This allows headers
@@ -419,7 +439,7 @@ class HeaderMap(CaseInsensitiveDict):
Values are header values (decoded according to :rfc:`2047` if necessary).
"""
protocol=(1, 1)
protocol = (1, 1)
encodings = ["ISO-8859-1"]
# Someday, when http-bis is done, this will probably get dropped
@@ -441,34 +461,42 @@ class HeaderMap(CaseInsensitiveDict):
def output(self):
"""Transform self into a list of (name, value) tuples."""
header_list = []
for k, v in self.items():
return list(self.encode_header_items(self.items()))
def encode_header_items(cls, header_items):
"""
Prepare the sequence of name, value tuples into a form suitable for
transmitting on the wire for HTTP.
"""
for k, v in header_items:
if isinstance(k, unicodestr):
k = self.encode(k)
k = cls.encode(k)
if not isinstance(v, basestring):
v = str(v)
if isinstance(v, unicodestr):
v = self.encode(v)
v = cls.encode(v)
# See header_translate_* constants above.
# Replace only if you really know what you're doing.
k = k.translate(header_translate_table, header_translate_deletechars)
v = v.translate(header_translate_table, header_translate_deletechars)
k = k.translate(header_translate_table,
header_translate_deletechars)
v = v.translate(header_translate_table,
header_translate_deletechars)
header_list.append((k, v))
return header_list
yield (k, v)
encode_header_items = classmethod(encode_header_items)
def encode(self, v):
def encode(cls, v):
"""Return the given header name or value, encoded for HTTP output."""
for enc in self.encodings:
for enc in cls.encodings:
try:
return v.encode(enc)
except UnicodeEncodeError:
continue
if self.protocol == (1, 1) and self.use_rfc_2047:
if cls.protocol == (1, 1) and cls.use_rfc_2047:
# Encode RFC-2047 TEXT
# (e.g. u"\u8200" -> "=?utf-8?b?6IiA?=").
# We do our own here instead of using the email module
@@ -479,10 +507,12 @@ class HeaderMap(CaseInsensitiveDict):
raise ValueError("Could not encode header part %r using "
"any of the encodings %r." %
(v, self.encodings))
(v, cls.encodings))
encode = classmethod(encode)
class Host(object):
"""An internet address.
name

21
lib/cherrypy/lib/jsontools.py
View File

@@ -1,6 +1,6 @@
import sys
import cherrypy
from cherrypy._cpcompat import basestring, ntou, json, json_encode, json_decode
from cherrypy._cpcompat import basestring, ntou, json_encode, json_decode
def json_processor(entity):
"""Read application/json data into request.json."""
@@ -13,8 +13,9 @@ def json_processor(entity):
except ValueError:
raise cherrypy.HTTPError(400, 'Invalid JSON document')
def json_in(content_type=[ntou('application/json'), ntou('text/javascript')],
force=True, debug=False, processor = json_processor):
force=True, debug=False, processor=json_processor):
"""Add a processor to parse JSON request entities:
The default processor places the parsed data into request.json.
@@ -57,11 +58,14 @@ def json_in(content_type=[ntou('application/json'), ntou('text/javascript')],
cherrypy.log('Adding body processor for %s' % ct, 'TOOLS.JSON_IN')
request.body.processors[ct] = processor
def json_handler(*args, **kwargs):
value = cherrypy.serving.request._json_inner_handler(*args, **kwargs)
return json_encode(value)
def json_out(content_type='application/json', debug=False, handler=json_handler):
def json_out(content_type='application/json', debug=False,
handler=json_handler):
"""Wrap request.handler to serialize its output to JSON. Sets Content-Type.
If the given content_type is None, the Content-Type response header
@@ -75,6 +79,11 @@ def json_out(content_type='application/json', debug=False, handler=json_handler)
package importable; otherwise, ValueError is raised during processing.
"""
request = cherrypy.serving.request
# request.handler may be set to None by e.g. the caching tool
# to signal to all components that a response body has already
# been attached, in which case we don't need to wrap anything.
if request.handler is None:
return
if debug:
cherrypy.log('Replacing %s with JSON handler' % request.handler,
'TOOLS.JSON_OUT')
@@ -82,6 +91,6 @@ def json_out(content_type='application/json', debug=False, handler=json_handler)
request.handler = handler
if content_type is not None:
if debug:
cherrypy.log('Setting Content-Type to %s' % content_type, 'TOOLS.JSON_OUT')
cherrypy.log('Setting Content-Type to %s' %
content_type, 'TOOLS.JSON_OUT')
cherrypy.serving.response.headers['Content-Type'] = content_type

147
lib/cherrypy/lib/lockfile.py Normal file
View File

@@ -0,0 +1,147 @@
"""
Platform-independent file locking. Inspired by and modeled after zc.lockfile.
"""
import os
try:
import msvcrt
except ImportError:
pass
try:
import fcntl
except ImportError:
pass
class LockError(Exception):
"Could not obtain a lock"
msg = "Unable to lock %r"
def __init__(self, path):
super(LockError, self).__init__(self.msg % path)
class UnlockError(LockError):
"Could not release a lock"
msg = "Unable to unlock %r"
# first, a default, naive locking implementation
class LockFile(object):
"""
A default, naive locking implementation. Always fails if the file
already exists.
"""
def __init__(self, path):
self.path = path
try:
fd = os.open(path, os.O_CREAT | os.O_WRONLY | os.O_EXCL)
except OSError:
raise LockError(self.path)
os.close(fd)
def release(self):
os.remove(self.path)
def remove(self):
pass
class SystemLockFile(object):
"""
An abstract base class for platform-specific locking.
"""
def __init__(self, path):
self.path = path
try:
# Open lockfile for writing without truncation:
self.fp = open(path, 'r+')
except IOError:
# If the file doesn't exist, IOError is raised; Use a+ instead.
# Note that there may be a race here. Multiple processes
# could fail on the r+ open and open the file a+, but only
# one will get the the lock and write a pid.
self.fp = open(path, 'a+')
try:
self._lock_file()
except:
self.fp.seek(1)
self.fp.close()
del self.fp
raise
self.fp.write(" %s\n" % os.getpid())
self.fp.truncate()
self.fp.flush()
def release(self):
if not hasattr(self, 'fp'):
return
self._unlock_file()
self.fp.close()
del self.fp
def remove(self):
"""
Attempt to remove the file
"""
try:
os.remove(self.path)
except:
pass
#@abc.abstract_method
# def _lock_file(self):
# """Attempt to obtain the lock on self.fp. Raise LockError if not
# acquired."""
def _unlock_file(self):
"""Attempt to obtain the lock on self.fp. Raise UnlockError if not
released."""
class WindowsLockFile(SystemLockFile):
def _lock_file(self):
# Lock just the first byte
try:
msvcrt.locking(self.fp.fileno(), msvcrt.LK_NBLCK, 1)
except IOError:
raise LockError(self.fp.name)
def _unlock_file(self):
try:
self.fp.seek(0)
msvcrt.locking(self.fp.fileno(), msvcrt.LK_UNLCK, 1)
except IOError:
raise UnlockError(self.fp.name)
if 'msvcrt' in globals():
LockFile = WindowsLockFile
class UnixLockFile(SystemLockFile):
def _lock_file(self):
flags = fcntl.LOCK_EX | fcntl.LOCK_NB
try:
fcntl.flock(self.fp.fileno(), flags)
except IOError:
raise LockError(self.fp.name)
# no need to implement _unlock_file, it will be unlocked on close()
if 'fcntl' in globals():
LockFile = UnixLockFile

47
lib/cherrypy/lib/locking.py Normal file
View File

@@ -0,0 +1,47 @@
import datetime
class NeverExpires(object):
def expired(self):
return False
class Timer(object):
"""
A simple timer that will indicate when an expiration time has passed.
"""
def __init__(self, expiration):
"Create a timer that expires at `expiration` (UTC datetime)"
self.expiration = expiration
@classmethod
def after(cls, elapsed):
"""
Return a timer that will expire after `elapsed` passes.
"""
return cls(datetime.datetime.utcnow() + elapsed)
def expired(self):
return datetime.datetime.utcnow() >= self.expiration
class LockTimeout(Exception):
"An exception when a lock could not be acquired before a timeout period"
class LockChecker(object):
"""
Keep track of the time and detect if a timeout has expired
"""
def __init__(self, session_id, timeout):
self.session_id = session_id
if timeout:
self.timer = Timer.after(timeout)
else:
self.timer = NeverExpires()
def expired(self):
if self.timer.expired():
raise LockTimeout(
"Timeout acquiring lock for %(session_id)s" % vars(self))
return False

34
lib/cherrypy/lib/profiler.py
View File

@@ -35,7 +35,8 @@ module from the command line, it will call ``serve()`` for you.
def new_func_strip_path(func_name):
"""Make profiler output more readable by adding ``__init__`` modules' parents"""
"""Make profiler output more readable by adding `__init__` modules' parents
"""
filename, line, name = func_name
if filename.endswith("__init__.py"):
return os.path.basename(filename[:-12]) + filename[-12:], line, name
@@ -49,14 +50,16 @@ except ImportError:
profile = None
pstats = None
import os, os.path
import os
import os.path
import sys
import warnings
from cherrypy._cpcompat import BytesIO
from cherrypy._cpcompat import StringIO
_count = 0
class Profiler(object):
def __init__(self, path=None):
@@ -85,7 +88,7 @@ class Profiler(object):
def stats(self, filename, sortby='cumulative'):
""":rtype stats(index): output of print_stats() for the given profile.
"""
sio = BytesIO()
sio = StringIO()
if sys.version_info >= (2, 5):
s = pstats.Stats(os.path.join(self.path, filename), stream=sio)
s.strip_dirs()
@@ -124,7 +127,8 @@ class Profiler(object):
runs = self.statfiles()
runs.sort()
for i in runs:
yield "<a href='report?filename=%s' target='main'>%s</a><br />" % (i, i)
yield "<a href='report?filename=%s' target='main'>%s</a><br />" % (
i, i)
menu.exposed = True
def report(self, filename):
@@ -142,14 +146,15 @@ class ProfileAggregator(Profiler):
self.count = _count = _count + 1
self.profiler = profile.Profile()
def run(self, func, *args):
def run(self, func, *args, **params):
path = os.path.join(self.path, "cp_%04d.prof" % self.count)
result = self.profiler.runcall(func, *args)
result = self.profiler.runcall(func, *args, **params)
self.profiler.dump_stats(path)
return result
class make_app:
def __init__(self, nextapp, path=None, aggregate=False):
"""Make a WSGI middleware app which wraps 'nextapp' with profiling.
@@ -167,9 +172,11 @@ class make_app:
"""
if profile is None or pstats is None:
msg = ("Your installation of Python does not have a profile module. "
"If you're on Debian, try `sudo apt-get install python-profiler`. "
"See http://www.cherrypy.org/wiki/ProfilingOnDebian for details.")
msg = ("Your installation of Python does not have a profile "
"module. If you're on Debian, try "
"`sudo apt-get install python-profiler`. "
"See http://www.cherrypy.org/wiki/ProfilingOnDebian "
"for details.")
warnings.warn(msg)
self.nextapp = nextapp
@@ -191,8 +198,10 @@ class make_app:
def serve(path=None, port=8080):
if profile is None or pstats is None:
msg = ("Your installation of Python does not have a profile module. "
"If you're on Debian, try `sudo apt-get install python-profiler`. "
"See http://www.cherrypy.org/wiki/ProfilingOnDebian for details.")
"If you're on Debian, try "
"`sudo apt-get install python-profiler`. "
"See http://www.cherrypy.org/wiki/ProfilingOnDebian "
"for details.")
warnings.warn(msg)
import cherrypy
@@ -205,4 +214,3 @@ def serve(path=None, port=8080):
if __name__ == "__main__":
serve(*tuple(sys.argv[1:]))

48
lib/cherrypy/lib/reprconf.py
View File

@@ -44,6 +44,7 @@ except ImportError:
import operator as _operator
import sys
def as_dict(config):
"""Return a dict from 'config' whether it is a dict, file, or filename."""
if isinstance(config, basestring):
@@ -54,6 +55,7 @@ def as_dict(config):
class NamespaceSet(dict):
"""A dict of config namespace names and handlers.
Each config entry should begin with a namespace name; the corresponding
@@ -129,6 +131,7 @@ class NamespaceSet(dict):
class Config(dict):
"""A dict-like set of configuration data, with defaults and namespaces.
May take a file, filename, or dict.
@@ -180,6 +183,7 @@ class Config(dict):
class Parser(ConfigParser):
"""Sub-class of ConfigParser that keeps the case of options and that
raises an exception if the file cannot be read.
"""
@@ -260,13 +264,31 @@ class _Builder2:
return expr[subs]
def build_CallFunc(self, o):
children = map(self.build, o.getChildren())
callee = children.pop(0)
kwargs = children.pop() or {}
starargs = children.pop() or ()
args = tuple(children) + tuple(starargs)
children = o.getChildren()
# Build callee from first child
callee = self.build(children[0])
# Build args and kwargs from remaining children
args = []
kwargs = {}
for child in children[1:]:
class_name = child.__class__.__name__
# None is ignored
if class_name == 'NoneType':
continue
# Keywords become kwargs
if class_name == 'Keyword':
kwargs.update(self.build(child))
# Everything else becomes args
else :
args.append(self.build(child))
return callee(*args, **kwargs)
def build_Keyword(self, o):
key, value_obj = o.getChildren()
value = self.build(value_obj)
kw_dict = {key: value}
return kw_dict
def build_List(self, o):
return map(self.build, o.getChildren())
@@ -415,6 +437,9 @@ class _Builder3:
raise TypeError("unrepr could not resolve the name %s" % repr(name))
def build_NameConstant(self, o):
return o.value
def build_UnaryOp(self, o):
op, operand = map(self.build, [o.op, o.operand])
return op(operand)
@@ -454,14 +479,9 @@ def unrepr(s):
def modules(modulePath):
"""Load a module and retrieve a reference to that module."""
try:
mod = sys.modules[modulePath]
if mod is None:
raise KeyError()
except KeyError:
# The last [''] is important.
mod = __import__(modulePath, globals(), locals(), [''])
return mod
__import__(modulePath)
return sys.modules[modulePath]
def attributes(full_attribute_name):
"""Load a module and retrieve an attribute of that module."""
@@ -481,5 +501,3 @@ def attributes(full_attribute_name):
# Return a reference to the attribute.
return attr

225
lib/cherrypy/lib/sessions.py
View File

@@ -8,10 +8,11 @@ You need to edit your config file to use sessions. Here's an example::
tools.sessions.storage_path = "/home/site/sessions"
tools.sessions.timeout = 60
This sets the session to be stored in files in the directory /home/site/sessions,
and the session timeout to 60 minutes. If you omit ``storage_type`` the sessions
will be saved in RAM. ``tools.sessions.on`` is the only required line for
working sessions, the rest are optional.
This sets the session to be stored in files in the directory
/home/site/sessions, and the session timeout to 60 minutes. If you omit
``storage_type`` the sessions will be saved in RAM.
``tools.sessions.on`` is the only required line for working sessions,
the rest are optional.
By default, the session ID is passed in a cookie, so the client's browser must
have cookies enabled for your site.
@@ -25,9 +26,14 @@ Locking sessions
================
By default, the ``'locking'`` mode of sessions is ``'implicit'``, which means
the session is locked early and unlocked late. If you want to control when the
session data is locked and unlocked, set ``tools.sessions.locking = 'explicit'``.
Then call ``cherrypy.session.acquire_lock()`` and ``cherrypy.session.release_lock()``.
the session is locked early and unlocked late. Be mindful of this default mode
for any requests that take a long time to process (streaming responses,
expensive calculations, database lookups, API calls, etc), as other concurrent
requests that also utilize sessions will hang until the session is unlocked.
If you want to control when the session data is locked and unlocked,
set ``tools.sessions.locking = 'explicit'``. Then call
``cherrypy.session.acquire_lock()`` and ``cherrypy.session.release_lock()``.
Regardless of which mode you use, the session is guaranteed to be unlocked when
the request is complete.
@@ -83,23 +89,25 @@ On the other extreme, some users report Firefox sending cookies after their
expiration date, although this was on a system with an inaccurate system time.
Maybe FF doesn't trust system time.
"""
import sys
import datetime
import os
import random
import time
import threading
import types
from warnings import warn
import cherrypy
from cherrypy._cpcompat import copyitems, pickle, random20, unicodestr
from cherrypy.lib import httputil
from cherrypy.lib import lockfile
from cherrypy.lib import locking
from cherrypy.lib import is_iterator
missing = object()
class Session(object):
"""A CherryPy dict-like Session object (one per request)."""
_id = None
@@ -109,6 +117,7 @@ class Session(object):
def _get_id(self):
return self._id
def _set_id(self, value):
self._id = value
for o in self.id_observers:
@@ -145,7 +154,10 @@ class Session(object):
True if the application called session.regenerate(). This is not set by
internal calls to regenerate the session id."""
debug=False
debug = False
"If True, log debug information."
# --------------------- Session management methods --------------------- #
def __init__(self, id=None, **kwargs):
self.id_observers = []
@@ -162,12 +174,15 @@ class Session(object):
self._regenerate()
else:
self.id = id
if not self._exists():
if self._exists():
if self.debug:
cherrypy.log('Set id to %s.' % id, 'TOOLS.SESSIONS')
else:
if self.debug:
cherrypy.log('Expired or malicious session %r; '
'making a new one' % id, 'TOOLS.SESSIONS')
# Expired or malicious session. Make a new one.
# See http://www.cherrypy.org/ticket/709.
# See https://bitbucket.org/cherrypy/cherrypy/issue/709.
self.id = None
self.missing = True
self._regenerate()
@@ -187,11 +202,18 @@ class Session(object):
def _regenerate(self):
if self.id is not None:
if self.debug:
cherrypy.log(
'Deleting the existing session %r before '
'regeneration.' % self.id,
'TOOLS.SESSIONS')
self.delete()
old_session_was_locked = self.locked
if old_session_was_locked:
self.release_lock()
if self.debug:
cherrypy.log('Old lock released.', 'TOOLS.SESSIONS')
self.id = None
while self.id is None:
@@ -199,9 +221,14 @@ class Session(object):
# Assert that the generated id is not already stored.
if self._exists():
self.id = None
if self.debug:
cherrypy.log('Set id to generated %s.' % self.id,
'TOOLS.SESSIONS')
if old_session_was_locked:
self.acquire_lock()
if self.debug:
cherrypy.log('Regenerated lock acquired.', 'TOOLS.SESSIONS')
def clean_up(self):
"""Clean up expired sessions."""
@@ -217,17 +244,24 @@ class Session(object):
# If session data has never been loaded then it's never been
# accessed: no need to save it
if self.loaded:
t = datetime.timedelta(seconds = self.timeout * 60)
t = datetime.timedelta(seconds=self.timeout * 60)
expiration_time = self.now() + t
if self.debug:
cherrypy.log('Saving with expiry %s' % expiration_time,
cherrypy.log('Saving session %r with expiry %s' %
(self.id, expiration_time),
'TOOLS.SESSIONS')
self._save(expiration_time)
else:
if self.debug:
cherrypy.log(
'Skipping save of session %r (no session loaded).' %
self.id, 'TOOLS.SESSIONS')
finally:
if self.locked:
# Always release the lock if the user didn't release it
self.release_lock()
if self.debug:
cherrypy.log('Lock released after save.', 'TOOLS.SESSIONS')
def load(self):
"""Copy stored session data into this session instance."""
@@ -235,9 +269,13 @@ class Session(object):
# data is either None or a tuple (session_data, expiration_time)
if data is None or data[1] < self.now():
if self.debug:
cherrypy.log('Expired session, flushing data', 'TOOLS.SESSIONS')
cherrypy.log('Expired session %r, flushing data.' % self.id,
'TOOLS.SESSIONS')
self._data = {}
else:
if self.debug:
cherrypy.log('Data loaded for session %r.' % self.id,
'TOOLS.SESSIONS')
self._data = data[0]
self.loaded = True
@@ -245,7 +283,7 @@ class Session(object):
# The instances are created and destroyed per-request.
cls = self.__class__
if self.clean_freq and not cls.clean_thread:
# clean_up is in instancemethod and not a classmethod,
# clean_up is an instancemethod and not a classmethod,
# so that tool config can be accessed inside the method.
t = cherrypy.process.plugins.Monitor(
cherrypy.engine, self.clean_up, self.clean_freq * 60,
@@ -253,21 +291,31 @@ class Session(object):
t.subscribe()
cls.clean_thread = t
t.start()
if self.debug:
cherrypy.log('Started cleanup thread.', 'TOOLS.SESSIONS')
def delete(self):
"""Delete stored session data."""
self._delete()
if self.debug:
cherrypy.log('Deleted session %s.' % self.id,
'TOOLS.SESSIONS')
# -------------------- Application accessor methods -------------------- #
def __getitem__(self, key):
if not self.loaded: self.load()
if not self.loaded:
self.load()
return self._data[key]
def __setitem__(self, key, value):
if not self.loaded: self.load()
if not self.loaded:
self.load()
self._data[key] = value
def __delitem__(self, key):
if not self.loaded: self.load()
if not self.loaded:
self.load()
del self._data[key]
def pop(self, key, default=missing):
@@ -275,55 +323,65 @@ class Session(object):
If key is not found, default is returned if given,
otherwise KeyError is raised.
"""
if not self.loaded: self.load()
if not self.loaded:
self.load()
if default is missing:
return self._data.pop(key)
else:
return self._data.pop(key, default)
def __contains__(self, key):
if not self.loaded: self.load()
if not self.loaded:
self.load()
return key in self._data
if hasattr({}, 'has_key'):
def has_key(self, key):
"""D.has_key(k) -> True if D has a key k, else False."""
if not self.loaded: self.load()
if not self.loaded:
self.load()
return key in self._data
def get(self, key, default=None):
"""D.get(k[,d]) -> D[k] if k in D, else d. d defaults to None."""
if not self.loaded: self.load()
if not self.loaded:
self.load()
return self._data.get(key, default)
def update(self, d):
"""D.update(E) -> None. Update D from E: for k in E: D[k] = E[k]."""
if not self.loaded: self.load()
if not self.loaded:
self.load()
self._data.update(d)
def setdefault(self, key, default=None):
"""D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D."""
if not self.loaded: self.load()
if not self.loaded:
self.load()
return self._data.setdefault(key, default)
def clear(self):
"""D.clear() -> None. Remove all items from D."""
if not self.loaded: self.load()
if not self.loaded:
self.load()
self._data.clear()
def keys(self):
"""D.keys() -> list of D's keys."""
if not self.loaded: self.load()
if not self.loaded:
self.load()
return self._data.keys()
def items(self):
"""D.items() -> list of D's (key, value) pairs, as 2-tuples."""
if not self.loaded: self.load()
if not self.loaded:
self.load()
return self._data.items()
def values(self):
"""D.values() -> list of D's values."""
if not self.loaded: self.load()
if not self.loaded:
self.load()
return self._data.values()
@@ -335,22 +393,26 @@ class RamSession(Session):
def clean_up(self):
"""Clean up expired sessions."""
now = self.now()
for id, (data, expiration_time) in copyitems(self.cache):
for _id, (data, expiration_time) in copyitems(self.cache):
if expiration_time <= now:
try:
del self.cache[id]
del self.cache[_id]
except KeyError:
pass
try:
del self.locks[id]
if self.locks[_id].acquire(blocking=False):
lock = self.locks.pop(_id)
lock.release()
except KeyError:
pass
# added to remove obsolete lock objects
for id in list(self.locks):
if id not in self.cache:
self.locks.pop(id, None)
for _id in list(self.locks):
if _id not in self.cache and self.locks[_id].acquire(blocking=False):
lock = self.locks.pop(_id)
lock.release()
def _exists(self):
return self.id in self.cache
@@ -380,6 +442,7 @@ class RamSession(Session):
class FileSession(Session):
"""Implementation of the File backend for sessions
storage_path
@@ -387,6 +450,10 @@ class FileSession(Session):
will be saved as pickle.dump(data, expiration_time) in its own file;
the filename will be self.SESSION_PREFIX + self.id.
lock_timeout
A timedelta or numeric seconds indicating how long
to block acquiring a lock. If None (default), acquiring a lock
will block indefinitely.
"""
SESSION_PREFIX = 'session-'
@@ -396,8 +463,17 @@ class FileSession(Session):
def __init__(self, id=None, **kwargs):
# The 'storage_path' arg is required for file-based sessions.
kwargs['storage_path'] = os.path.abspath(kwargs['storage_path'])
kwargs.setdefault('lock_timeout', None)
Session.__init__(self, id=id, **kwargs)
# validate self.lock_timeout
if isinstance(self.lock_timeout, (int, float)):
self.lock_timeout = datetime.timedelta(seconds=self.lock_timeout)
if not isinstance(self.lock_timeout, (datetime.timedelta, type(None))):
raise ValueError("Lock timeout must be numeric seconds or "
"a timedelta instance.")
def setup(cls, **kwargs):
"""Set up the storage system for file-based sessions.
@@ -409,17 +485,6 @@ class FileSession(Session):
for k, v in kwargs.items():
setattr(cls, k, v)
# Warn if any lock files exist at startup.
lockfiles = [fname for fname in os.listdir(cls.storage_path)
if (fname.startswith(cls.SESSION_PREFIX)
and fname.endswith(cls.LOCK_SUFFIX))]
if lockfiles:
plural = ('', 's')[len(lockfiles) > 1]
warn("%s session lockfile%s found at startup. If you are "
"only running one process, then you may need to "
"manually delete the lockfiles found at %r."
% (len(lockfiles), plural, cls.storage_path))
setup = classmethod(setup)
def _get_file_path(self):
@@ -433,6 +498,8 @@ class FileSession(Session):
return os.path.exists(path)
def _load(self, path=None):
assert self.locked, ("The session load without being locked. "
"Check your tools' priority levels.")
if path is None:
path = self._get_file_path()
try:
@@ -442,9 +509,15 @@ class FileSession(Session):
finally:
f.close()
except (IOError, EOFError):
e = sys.exc_info()[1]
if self.debug:
cherrypy.log("Error loading the session pickle: %s" %
e, 'TOOLS.SESSIONS')
return None
def _save(self, expiration_time):
assert self.locked, ("The session was saved without being locked. "
"Check your tools' priority levels.")
f = open(self._get_file_path(), "wb")
try:
pickle.dump((self._data, expiration_time), f, self.pickle_protocol)
@@ -452,6 +525,8 @@ class FileSession(Session):
f.close()
def _delete(self):
assert self.locked, ("The session deletion without being locked. "
"Check your tools' priority levels.")
try:
os.unlink(self._get_file_path())
except OSError:
@@ -462,21 +537,22 @@ class FileSession(Session):
if path is None:
path = self._get_file_path()
path += self.LOCK_SUFFIX
while True:
checker = locking.LockChecker(self.id, self.lock_timeout)
while not checker.expired():
try:
lockfd = os.open(path, os.O_CREAT|os.O_WRONLY|os.O_EXCL)
except OSError:
self.lock = lockfile.LockFile(path)
except lockfile.LockError:
time.sleep(0.1)
else:
os.close(lockfd)
break
self.locked = True
if self.debug:
cherrypy.log('Lock acquired.', 'TOOLS.SESSIONS')
def release_lock(self, path=None):
"""Release the lock on the currently-loaded session data."""
if path is None:
path = self._get_file_path()
os.unlink(path + self.LOCK_SUFFIX)
self.lock.release()
self.lock.remove()
self.locked = False
def clean_up(self):
@@ -485,11 +561,18 @@ class FileSession(Session):
# Iterate over all session files in self.storage_path
for fname in os.listdir(self.storage_path):
if (fname.startswith(self.SESSION_PREFIX)
and not fname.endswith(self.LOCK_SUFFIX)):
and not fname.endswith(self.LOCK_SUFFIX)):
# We have a session file: lock and load it and check
# if it's expired. If it fails, nevermind.
path = os.path.join(self.storage_path, fname)
self.acquire_lock(path)
if self.debug:
# This is a bit of a hack, since we're calling clean_up
# on the first instance rather than the entire class,
# so depending on whether you have "debug" set on the
# path of the first session called, this may not run.
cherrypy.log('Cleanup lock acquired.', 'TOOLS.SESSIONS')
try:
contents = self._load(path)
# _load returns None on IOError
@@ -509,6 +592,7 @@ class FileSession(Session):
class PostgresqlSession(Session):
""" Implementation of the PostgreSQL backend for sessions. It assumes
a table like this::
@@ -578,6 +662,8 @@ class PostgresqlSession(Session):
self.locked = True
self.cursor.execute('select id from session where id=%s for update',
(self.id,))
if self.debug:
cherrypy.log('Lock acquired.', 'TOOLS.SESSIONS')
def release_lock(self):
"""Release the lock on the currently-loaded session data."""
@@ -618,6 +704,7 @@ class MemcachedSession(Session):
def _get_id(self):
return self._id
def _set_id(self, value):
# This encode() call is where we differ from the superclass.
# Memcache keys MUST be byte strings, not unicode.
@@ -649,7 +736,8 @@ class MemcachedSession(Session):
self.mc_lock.acquire()
try:
if not self.cache.set(self.id, (self._data, expiration_time), td):
raise AssertionError("Session data for id %r not set." % self.id)
raise AssertionError(
"Session data for id %r not set." % self.id)
finally:
self.mc_lock.release()
@@ -660,6 +748,8 @@ class MemcachedSession(Session):
"""Acquire an exclusive lock on the currently-loaded session data."""
self.locked = True
self.locks.setdefault(self.id, threading.RLock()).acquire()
if self.debug:
cherrypy.log('Lock acquired.', 'TOOLS.SESSIONS')
def release_lock(self):
"""Release the lock on the currently-loaded session data."""
@@ -693,17 +783,20 @@ def save():
else:
# If the body is not being streamed, we save the data now
# (so we can release the lock).
if isinstance(response.body, types.GeneratorType):
if is_iterator(response.body):
response.collapse_body()
cherrypy.session.save()
save.failsafe = True
def close():
"""Close the session object for this request."""
sess = getattr(cherrypy.serving, "session", None)
if getattr(sess, "locked", False):
# If the session is still locked we release the lock
sess.release_lock()
if sess.debug:
cherrypy.log('Lock released on close.', 'TOOLS.SESSIONS')
close.failsafe = True
close.priority = 90
@@ -787,6 +880,7 @@ def init(storage_type='ram', path=None, path_header=None, name='session_id',
kwargs['clean_freq'] = clean_freq
cherrypy.serving.session = sess = storage_class(id, **kwargs)
sess.debug = debug
def update_cookie(id):
"""Update the cookie every time the session id changes."""
cherrypy.serving.response.cookie[name] = id
@@ -841,8 +935,11 @@ def set_response_cookie(path=None, path_header=None, name='session_id',
# Set response cookie
cookie = cherrypy.serving.response.cookie
cookie[name] = cherrypy.serving.session.id
cookie[name]['path'] = (path or cherrypy.serving.request.headers.get(path_header)
or '/')
cookie[name]['path'] = (
path or
cherrypy.serving.request.headers.get(path_header) or
'/'
)
# We'd like to use the "max-age" param as indicated in
# http://www.faqs.org/rfcs/rfc2109.html but IE doesn't
@@ -861,11 +958,11 @@ def set_response_cookie(path=None, path_header=None, name='session_id',
raise ValueError("The httponly cookie token is not supported.")
cookie[name]['httponly'] = 1
def expire():
"""Expire the current session cookie."""
name = cherrypy.serving.request.config.get('tools.sessions.name', 'session_id')
name = cherrypy.serving.request.config.get(
'tools.sessions.name', 'session_id')
one_year = 60 * 60 * 24 * 365
e = time.time() - one_year
cherrypy.serving.response.cookie[name]['expires'] = httputil.HTTPDate(e)

69
lib/cherrypy/lib/static.py
View File

@@ -1,26 +1,27 @@
import os
import re
import stat
import mimetypes
try:
from io import UnsupportedOperation
except ImportError:
UnsupportedOperation = object()
import logging
import mimetypes
mimetypes.init()
mimetypes.types_map['.dwg']='image/x-dwg'
mimetypes.types_map['.ico']='image/x-icon'
mimetypes.types_map['.bz2']='application/x-bzip2'
mimetypes.types_map['.gz']='application/x-gzip'
import os
import re
import stat
import time
import cherrypy
from cherrypy._cpcompat import ntob, unquote
from cherrypy.lib import cptools, httputil, file_generator_limited
def serve_file(path, content_type=None, disposition=None, name=None, debug=False):
mimetypes.init()
mimetypes.types_map['.dwg'] = 'image/x-dwg'
mimetypes.types_map['.ico'] = 'image/x-icon'
mimetypes.types_map['.bz2'] = 'application/x-bzip2'
mimetypes.types_map['.gz'] = 'application/x-gzip'
def serve_file(path, content_type=None, disposition=None, name=None,
debug=False):
"""Set status, headers, and body in order to serve the given path.
The Content-Type header will be set to the content_type arg, if provided.
@@ -92,6 +93,7 @@ def serve_file(path, content_type=None, disposition=None, name=None, debug=False
fileobj = open(path, 'rb')
return _serve_fileobj(fileobj, content_type, content_length, debug=debug)
def serve_fileobj(fileobj, content_type=None, disposition=None, name=None,
debug=False):
"""Set status, headers, and body in order to serve the given file object.
@@ -145,6 +147,7 @@ def serve_fileobj(fileobj, content_type=None, disposition=None, name=None,
return _serve_fileobj(fileobj, content_type, content_length, debug=debug)
def _serve_fileobj(fileobj, content_type, content_length, debug=False):
"""Internal. Set response.body to the given file object, perhaps ranged."""
response = cherrypy.serving.response
@@ -156,7 +159,8 @@ def _serve_fileobj(fileobj, content_type, content_length, debug=False):
r = httputil.get_ranges(request.headers.get('Range'), content_length)
if r == []:
response.headers['Content-Range'] = "bytes */%s" % content_length
message = "Invalid Range (first-byte-pos greater than Content-Length)"
message = ("Invalid Range (first-byte-pos greater than "
"Content-Length)")
if debug:
cherrypy.log(message, 'TOOLS.STATIC')
raise cherrypy.HTTPError(416, message)
@@ -169,8 +173,9 @@ def _serve_fileobj(fileobj, content_type, content_length, debug=False):
stop = content_length
r_len = stop - start
if debug:
cherrypy.log('Single part; start: %r, stop: %r' % (start, stop),
'TOOLS.STATIC')
cherrypy.log(
'Single part; start: %r, stop: %r' % (start, stop),
'TOOLS.STATIC')
response.status = "206 Partial Content"
response.headers['Content-Range'] = (
"bytes %s-%s/%s" % (start, stop - 1, content_length))
@@ -182,11 +187,11 @@ def _serve_fileobj(fileobj, content_type, content_length, debug=False):
response.status = "206 Partial Content"
try:
# Python 3
from email.generator import _make_boundary as choose_boundary
from email.generator import _make_boundary as make_boundary
except ImportError:
# Python 2
from mimetools import choose_boundary
boundary = choose_boundary()
from mimetools import choose_boundary as make_boundary
boundary = make_boundary()
ct = "multipart/byteranges; boundary=%s" % boundary
response.headers['Content-Type'] = ct
if "Content-Length" in response.headers:
@@ -199,14 +204,20 @@ def _serve_fileobj(fileobj, content_type, content_length, debug=False):
for start, stop in r:
if debug:
cherrypy.log('Multipart; start: %r, stop: %r' % (start, stop),
'TOOLS.STATIC')
cherrypy.log(
'Multipart; start: %r, stop: %r' % (
start, stop),
'TOOLS.STATIC')
yield ntob("--" + boundary, 'ascii')
yield ntob("\r\nContent-type: %s" % content_type, 'ascii')
yield ntob("\r\nContent-range: bytes %s-%s/%s\r\n\r\n"
% (start, stop - 1, content_length), 'ascii')
yield ntob("\r\nContent-type: %s" % content_type,
'ascii')
yield ntob(
"\r\nContent-range: bytes %s-%s/%s\r\n\r\n" % (
start, stop - 1, content_length),
'ascii')
fileobj.seek(start)
for chunk in file_generator_limited(fileobj, stop-start):
gen = file_generator_limited(fileobj, stop - start)
for chunk in gen:
yield chunk
yield ntob("\r\n")
# Final boundary
@@ -226,6 +237,7 @@ def _serve_fileobj(fileobj, content_type, content_length, debug=False):
response.body = fileobj
return response.body
def serve_download(path, name=None):
"""Serve 'path' as an application/x-download attachment."""
# This is such a common idiom I felt it deserved its own wrapper.
@@ -252,6 +264,7 @@ def _attempt(filename, content_types, debug=False):
cherrypy.log('NotFound', 'TOOLS.STATICFILE')
return False
def staticdir(section, dir, root="", match="", content_types=None, index="",
debug=False):
"""Serve a static resource from the given (root +) dir.
@@ -314,7 +327,7 @@ def staticdir(section, dir, root="", match="", content_types=None, index="",
# have ".." or similar uplevel attacks in it. Check that the final
# filename is a child of dir.
if not os.path.normpath(filename).startswith(os.path.normpath(dir)):
raise cherrypy.HTTPError(403) # Forbidden
raise cherrypy.HTTPError(403) # Forbidden
handled = _attempt(filename, content_types)
if not handled:
@@ -325,6 +338,7 @@ def staticdir(section, dir, root="", match="", content_types=None, index="",
request.is_index = filename[-1] in (r"\/")
return handled
def staticfile(filename, root=None, match="", content_types=None, debug=False):
"""Serve a static resource from the given (root +) filename.
@@ -354,7 +368,8 @@ def staticfile(filename, root=None, match="", content_types=None, debug=False):
# If filename is relative, make absolute using "root".
if not os.path.isabs(filename):
if not root:
msg = "Static tool requires an absolute filename (got '%s')." % filename
msg = "Static tool requires an absolute filename (got '%s')." % (
filename,)
if debug:
cherrypy.log(msg, 'TOOLS.STATICFILE')
raise ValueError(msg)

4
lib/cherrypy/lib/xmlrpcutil.py
View File

@@ -3,6 +3,7 @@ import sys
import cherrypy
from cherrypy._cpcompat import ntob
def get_xmlrpclib():
try:
import xmlrpc.client as x
@@ -10,6 +11,7 @@ def get_xmlrpclib():
import xmlrpclib as x
return x
def process_body():
"""Return (params, method) from request body."""
try:
@@ -48,8 +50,8 @@ def respond(body, encoding='utf-8', allow_none=0):
encoding=encoding,
allow_none=allow_none))
def on_error(*args, **kwargs):
body = str(sys.exc_info()[1])
xmlrpclib = get_xmlrpclib()
_set_response(xmlrpclib.dumps(xmlrpclib.Fault(1, body)))

108
lib/cherrypy/process/plugins.py
View File

@@ -7,7 +7,8 @@ import sys
import time
import threading
from cherrypy._cpcompat import basestring, get_daemon, get_thread_ident, ntob, set
from cherrypy._cpcompat import basestring, get_daemon, get_thread_ident
from cherrypy._cpcompat import ntob, set, Timer, SetDaemonProperty
# _module__file__base is used by Autoreload to make
# absolute any filenames retrieved from sys.modules which are not
@@ -19,8 +20,8 @@ from cherrypy._cpcompat import basestring, get_daemon, get_thread_ident, ntob, s
# changes the current directory by executing os.chdir(), then the next time
# Autoreload runs, it will not be able to find any filenames which are
# not absolute paths, because the current directory is not the same as when the
# module was first imported. Autoreload will then wrongly conclude the file has
# "changed", and initiate the shutdown/re-exec sequence.
# module was first imported. Autoreload will then wrongly conclude the file
# has "changed", and initiate the shutdown/re-exec sequence.
# See ticket #917.
# For this workaround to have a decent probability of success, this module
# needs to be imported as early as possible, before the app has much chance
@@ -29,10 +30,12 @@ _module__file__base = os.getcwd()
class SimplePlugin(object):
"""Plugin base class which auto-subscribes methods for known channels."""
bus = None
"""A :class:`Bus <cherrypy.process.wspbus.Bus>`, usually cherrypy.engine."""
"""A :class:`Bus <cherrypy.process.wspbus.Bus>`, usually cherrypy.engine.
"""
def __init__(self, bus):
self.bus = bus
@@ -54,8 +57,8 @@ class SimplePlugin(object):
self.bus.unsubscribe(channel, method)
class SignalHandler(object):
"""Register bus channels (and listeners) for system signals.
You can modify what signals your application listens for, and what it does
@@ -74,9 +77,9 @@ class SignalHandler(object):
if the process is attached to a TTY. This is because Unix window
managers tend to send SIGHUP to terminal windows when the user closes them.
Feel free to add signals which are not available on every platform. The
:class:`SignalHandler` will ignore errors raised from attempting to register
handlers for unknown signals.
Feel free to add signals which are not available on every platform.
The :class:`SignalHandler` will ignore errors raised from attempting
to register handlers for unknown signals.
"""
handlers = {}
@@ -187,15 +190,17 @@ class SignalHandler(object):
try:
import pwd, grp
import pwd
import grp
except ImportError:
pwd, grp = None, None
class DropPrivileges(SimplePlugin):
"""Drop privileges. uid/gid arguments not available on Windows.
Special thanks to Gavin Baker: http://antonym.org/node/100.
Special thanks to `Gavin Baker <http://antonym.org/2005/12/dropping-privileges-in-python.html>`_
"""
def __init__(self, bus, umask=None, uid=None, gid=None):
@@ -207,6 +212,7 @@ class DropPrivileges(SimplePlugin):
def _get_uid(self):
return self._uid
def _set_uid(self, val):
if val is not None:
if pwd is None:
@@ -217,10 +223,11 @@ class DropPrivileges(SimplePlugin):
val = pwd.getpwnam(val)[2]
self._uid = val
uid = property(_get_uid, _set_uid,
doc="The uid under which to run. Availability: Unix.")
doc="The uid under which to run. Availability: Unix.")
def _get_gid(self):
return self._gid
def _set_gid(self, val):
if val is not None:
if grp is None:
@@ -231,10 +238,11 @@ class DropPrivileges(SimplePlugin):
val = grp.getgrnam(val)[2]
self._gid = val
gid = property(_get_gid, _set_gid,
doc="The gid under which to run. Availability: Unix.")
doc="The gid under which to run. Availability: Unix.")
def _get_umask(self):
return self._umask
def _set_umask(self, val):
if val is not None:
try:
@@ -244,8 +252,11 @@ class DropPrivileges(SimplePlugin):
level=30)
val = None
self._umask = val
umask = property(_get_umask, _set_umask,
doc="""The default permission mode for newly created files and directories.
umask = property(
_get_umask,
_set_umask,
doc="""The default permission mode for newly created files and
directories.
Usually expressed in octal format, for example, ``0644``.
Availability: Unix, Windows.
@@ -299,6 +310,7 @@ class DropPrivileges(SimplePlugin):
class Daemonizer(SimplePlugin):
"""Daemonize the running script.
Use this with a Web Site Process Bus via::
@@ -368,7 +380,7 @@ class Daemonizer(SimplePlugin):
pid = os.fork()
if pid > 0:
self.bus.log('Forking twice.')
os._exit(0) # Exit second parent
os._exit(0) # Exit second parent
except OSError:
exc = sys.exc_info()[1]
sys.exit("%s: fork #2 failed: (%d) %s\n"
@@ -394,6 +406,7 @@ class Daemonizer(SimplePlugin):
class PIDFile(SimplePlugin):
"""Maintain a PID file via a WSPBus."""
def __init__(self, bus, pidfile):
@@ -406,7 +419,7 @@ class PIDFile(SimplePlugin):
if self.finalized:
self.bus.log('PID %r already written to %r.' % (pid, self.pidfile))
else:
open(self.pidfile, "wb").write(ntob("%s" % pid, 'utf8'))
open(self.pidfile, "wb").write(ntob("%s\n" % pid, 'utf8'))
self.bus.log('PID %r written to %r.' % (pid, self.pidfile))
self.finalized = True
start.priority = 70
@@ -421,14 +434,20 @@ class PIDFile(SimplePlugin):
pass
class PerpetualTimer(threading._Timer):
"""A responsive subclass of threading._Timer whose run() method repeats.
class PerpetualTimer(Timer):
"""A responsive subclass of threading.Timer whose run() method repeats.
Use this timer only when you really need a very interruptible timer;
this checks its 'finished' condition up to 20 times a second, which can
results in pretty high CPU usage
"""
def __init__(self, *args, **kwargs):
"Override parent constructor to allow 'bus' to be provided."
self.bus = kwargs.pop('bus', None)
super(PerpetualTimer, self).__init__(*args, **kwargs)
def run(self):
while True:
self.finished.wait(self.interval)
@@ -437,13 +456,16 @@ class PerpetualTimer(threading._Timer):
try:
self.function(*self.args, **self.kwargs)
except Exception:
self.bus.log("Error in perpetual timer thread function %r." %
self.function, level=40, traceback=True)
if self.bus:
self.bus.log(
"Error in perpetual timer thread function %r." %
self.function, level=40, traceback=True)
# Quit on first error to avoid massive logs.
raise
class BackgroundTask(threading.Thread):
class BackgroundTask(SetDaemonProperty, threading.Thread):
"""A subclass of threading.Thread whose run() method repeats.
Use this class for most repeating tasks. It uses time.sleep() to wait
@@ -462,6 +484,9 @@ class BackgroundTask(threading.Thread):
self.running = False
self.bus = bus
# default to daemonic
self.daemon = True
def cancel(self):
self.running = False
@@ -480,11 +505,9 @@ class BackgroundTask(threading.Thread):
# Quit on first error to avoid massive logs.
raise
def _set_daemon(self):
return True
class Monitor(SimplePlugin):
"""WSPBus listener to periodically run a callback in its own thread."""
callback = None
@@ -494,7 +517,9 @@ class Monitor(SimplePlugin):
"""The time in seconds between callback runs."""
thread = None
"""A :class:`BackgroundTask<cherrypy.process.plugins.BackgroundTask>` thread."""
"""A :class:`BackgroundTask<cherrypy.process.plugins.BackgroundTask>`
thread.
"""
def __init__(self, bus, callback, frequency=60, name=None):
SimplePlugin.__init__(self, bus)
@@ -509,7 +534,7 @@ class Monitor(SimplePlugin):
threadname = self.name or self.__class__.__name__
if self.thread is None:
self.thread = BackgroundTask(self.frequency, self.callback,
bus = self.bus)
bus=self.bus)
self.thread.setName(threadname)
self.thread.start()
self.bus.log("Started monitor thread %r." % threadname)
@@ -520,7 +545,8 @@ class Monitor(SimplePlugin):
def stop(self):
"""Stop our callback's background task thread."""
if self.thread is None:
self.bus.log("No thread running for %s." % self.name or self.__class__.__name__)
self.bus.log("No thread running for %s." %
self.name or self.__class__.__name__)
else:
if self.thread is not threading.currentThread():
name = self.thread.getName()
@@ -538,6 +564,7 @@ class Monitor(SimplePlugin):
class Autoreloader(Monitor):
"""Monitor which re-executes the process when files change.
This :ref:`plugin<plugins>` restarts the process (via :func:`os.execv`)
@@ -547,9 +574,9 @@ class Autoreloader(Monitor):
cherrypy.engine.autoreload.files.add(myFile)
If there are imported files you do *not* wish to monitor, you can adjust the
``match`` attribute, a regular expression. For example, to stop monitoring
cherrypy itself::
If there are imported files you do *not* wish to monitor, you can
adjust the ``match`` attribute, a regular expression. For example,
to stop monitoring cherrypy itself::
cherrypy.engine.autoreload.match = r'^(?!cherrypy).+'
@@ -583,15 +610,20 @@ class Autoreloader(Monitor):
def sysfiles(self):
"""Return a Set of sys.modules filenames to monitor."""
files = set()
for k, m in sys.modules.items():
for k, m in list(sys.modules.items()):
if re.match(self.match, k):
if hasattr(m, '__loader__') and hasattr(m.__loader__, 'archive'):
if (
hasattr(m, '__loader__') and
hasattr(m.__loader__, 'archive')
):
f = m.__loader__.archive
else:
f = getattr(m, '__file__', None)
if f is not None and not os.path.isabs(f):
# ensure absolute paths so a os.chdir() in the app doesn't break me
f = os.path.normpath(os.path.join(_module__file__base, f))
# ensure absolute paths so a os.chdir() in the app
# doesn't break me
f = os.path.normpath(
os.path.join(_module__file__base, f))
files.add(f)
return files
@@ -619,14 +651,17 @@ class Autoreloader(Monitor):
else:
if mtime is None or mtime > oldtime:
# The file has been deleted or modified.
self.bus.log("Restarting because %s changed." % filename)
self.bus.log("Restarting because %s changed." %
filename)
self.thread.cancel()
self.bus.log("Stopped thread %r." % self.thread.getName())
self.bus.log("Stopped thread %r." %
self.thread.getName())
self.bus.restart()
return
class ThreadManager(SimplePlugin):
"""Manager for HTTP request threads.
If you have control over thread creation and destruction, publish to
@@ -680,4 +715,3 @@ class ThreadManager(SimplePlugin):
self.bus.publish('stop_thread', i)
self.threads.clear()
graceful = stop

64
lib/cherrypy/process/servers.py
View File

@@ -13,7 +13,9 @@ protocols, etc.), you can manually register each one and then start them all
with engine.start::
s1 = ServerAdapter(cherrypy.engine, MyWSGIServer(host='0.0.0.0', port=80))
s2 = ServerAdapter(cherrypy.engine, another.HTTPServer(host='127.0.0.1', SSL=True))
s2 = ServerAdapter(cherrypy.engine,
another.HTTPServer(host='127.0.0.1',
SSL=True))
s1.subscribe()
s2.subscribe()
cherrypy.engine.start()
@@ -63,7 +65,7 @@ hello.py::
cherrypy.tree.mount(HelloWorld())
# CherryPy autoreload must be disabled for the flup server to work
cherrypy.config.update({'engine.autoreload_on':False})
cherrypy.config.update({'engine.autoreload.on':False})
Then run :doc:`/deployguide/cherryd` with the '-f' arg::
@@ -107,15 +109,17 @@ directive, configure your fastcgi script like the following::
} # end of $HTTP["url"] =~ "^/"
Please see `Lighttpd FastCGI Docs
<http://redmine.lighttpd.net/wiki/lighttpd/Docs:ModFastCGI>`_ for an explanation
of the possible configuration options.
<http://redmine.lighttpd.net/wiki/lighttpd/Docs:ModFastCGI>`_ for
an explanation of the possible configuration options.
"""
import sys
import time
import warnings
class ServerAdapter(object):
"""Adapter for an HTTP server.
If you need to start more than one HTTP server (to serve on multiple
@@ -149,8 +153,7 @@ class ServerAdapter(object):
if self.bind_addr is None:
on_what = "unknown interface (dynamic?)"
elif isinstance(self.bind_addr, tuple):
host, port = self.bind_addr
on_what = "%s:%s" % (host, port)
on_what = self._get_base()
else:
on_what = "socket file: %s" % self.bind_addr
@@ -176,6 +179,21 @@ class ServerAdapter(object):
self.bus.log("Serving on %s" % on_what)
start.priority = 75
def _get_base(self):
if not self.httpserver:
return ''
host, port = self.bind_addr
if getattr(self.httpserver, 'ssl_certificate', None):
scheme = "https"
if port != 443:
host += ":%s" % port
else:
scheme = "http"
if port != 80:
host += ":%s" % port
return "%s://%s" % (scheme, host)
def _start_http_thread(self):
"""HTTP servers MUST be running in new threads, so that the
main thread persists to receive KeyboardInterrupt's. If an
@@ -234,6 +252,7 @@ class ServerAdapter(object):
class FlupCGIServer(object):
"""Adapter for a flup.server.cgi.WSGIServer."""
def __init__(self, *args, **kwargs):
@@ -257,6 +276,7 @@ class FlupCGIServer(object):
class FlupFCGIServer(object):
"""Adapter for a flup.server.fcgi.WSGIServer."""
def __init__(self, *args, **kwargs):
@@ -296,11 +316,13 @@ class FlupFCGIServer(object):
# Forcibly stop the fcgi server main event loop.
self.fcgiserver._keepGoing = False
# Force all worker threads to die off.
self.fcgiserver._threadPool.maxSpare = self.fcgiserver._threadPool._idleCount
self.fcgiserver._threadPool.maxSpare = (
self.fcgiserver._threadPool._idleCount)
self.ready = False
class FlupSCGIServer(object):
"""Adapter for a flup.server.scgi.WSGIServer."""
def __init__(self, *args, **kwargs):
@@ -344,10 +366,12 @@ def client_host(server_host):
return '127.0.0.1'
if server_host in ('::', '::0', '::0.0.0.0'):
# :: is IN6ADDR_ANY, which should answer on localhost.
# ::0 and ::0.0.0.0 are non-canonical but common ways to write IN6ADDR_ANY.
# ::0 and ::0.0.0.0 are non-canonical but common
# ways to write IN6ADDR_ANY.
return '::1'
return server_host
def check_port(host, port, timeout=1.0):
"""Raise an error if the given port is not free on the given host."""
if not host:
@@ -364,7 +388,9 @@ def check_port(host, port, timeout=1.0):
socket.SOCK_STREAM)
except socket.gaierror:
if ':' in host:
info = [(socket.AF_INET6, socket.SOCK_STREAM, 0, "", (host, port, 0, 0))]
info = [(
socket.AF_INET6, socket.SOCK_STREAM, 0, "", (host, port, 0, 0)
)]
else:
info = [(socket.AF_INET, socket.SOCK_STREAM, 0, "", (host, port))]
@@ -378,18 +404,20 @@ def check_port(host, port, timeout=1.0):
s.settimeout(timeout)
s.connect((host, port))
s.close()
raise IOError("Port %s is in use on %s; perhaps the previous "
"httpserver did not shut down properly." %
(repr(port), repr(host)))
except socket.error:
if s:
s.close()
else:
raise IOError("Port %s is in use on %s; perhaps the previous "
"httpserver did not shut down properly." %
(repr(port), repr(host)))
# Feel free to increase these defaults on slow systems:
free_port_timeout = 0.1
occupied_port_timeout = 1.0
def wait_for_free_port(host, port, timeout=None):
"""Wait for the specified port to become free (drop requests)."""
if not host:
@@ -409,6 +437,7 @@ def wait_for_free_port(host, port, timeout=None):
raise IOError("Port %r not free on %r" % (port, host))
def wait_for_occupied_port(host, port, timeout=None):
"""Wait for the specified port to become active (receive requests)."""
if not host:
@@ -420,8 +449,17 @@ def wait_for_occupied_port(host, port, timeout=None):
try:
check_port(host, port, timeout=timeout)
except IOError:
# port is occupied
return
else:
time.sleep(timeout)
raise IOError("Port %r not bound on %r" % (port, host))
if host == client_host(host):
raise IOError("Port %r not bound on %r" % (port, host))
# On systems where a loopback interface is not available and the
# server is bound to all interfaces, it's difficult to determine
# whether the server is in fact occupying the port. In this case,
# just issue a warning and move on. See issue #1100.
msg = "Unable to verify that the server is bound on %r" % port
warnings.warn(msg)

8
lib/cherrypy/process/win32.py
View File

@@ -11,6 +11,7 @@ from cherrypy.process import wspbus, plugins
class ConsoleCtrlHandler(plugins.SimplePlugin):
"""A WSPBus plugin for handling Win32 console events (like Ctrl-C)."""
def __init__(self, bus):
@@ -68,6 +69,7 @@ class ConsoleCtrlHandler(plugins.SimplePlugin):
class Win32Bus(wspbus.Bus):
"""A Web Site Process Bus implementation for Win32.
Instead of time.sleep, this bus blocks using native win32event objects.
@@ -90,6 +92,7 @@ class Win32Bus(wspbus.Bus):
def _get_state(self):
return self._state
def _set_state(self, value):
self._state = value
event = self._get_state_event(value)
@@ -106,7 +109,8 @@ class Win32Bus(wspbus.Bus):
# Don't wait for an event that beat us to the punch ;)
if self.state not in state:
events = tuple([self._get_state_event(s) for s in state])
win32event.WaitForMultipleObjects(events, 0, win32event.INFINITE)
win32event.WaitForMultipleObjects(
events, 0, win32event.INFINITE)
else:
# Don't wait for an event that beat us to the punch ;)
if self.state != state:
@@ -115,6 +119,7 @@ class Win32Bus(wspbus.Bus):
class _ControlCodes(dict):
"""Control codes used to "signal" a service via ControlService.
User-defined control codes are in the range 128-255. We generally use
@@ -145,6 +150,7 @@ def signal_child(service, command):
class PyWebService(win32serviceutil.ServiceFramework):
"""Python Web Service."""
_svc_name_ = "Python Web Service"

32
lib/cherrypy/process/wspbus.py
View File

@@ -78,13 +78,16 @@ from cherrypy._cpcompat import set
# sys.executable is a relative-path, and/or cause other problems).
_startup_cwd = os.getcwd()
class ChannelFailures(Exception):
"""Exception raised when errors occur in a listener during Bus.publish()."""
"""Exception raised when errors occur in a listener during Bus.publish().
"""
delimiter = '\n'
def __init__(self, *args, **kwargs):
# Don't use 'super' here; Exceptions are old-style in Py2.4
# See http://www.cherrypy.org/ticket/959
# See https://bitbucket.org/cherrypy/cherrypy/issue/959
Exception.__init__(self, *args, **kwargs)
self._exceptions = list()
@@ -107,9 +110,13 @@ class ChannelFailures(Exception):
__nonzero__ = __bool__
# Use a flag to indicate the state of the bus.
class _StateEnum(object):
class State(object):
name = None
def __repr__(self):
return "states.%s" % self.name
@@ -137,6 +144,7 @@ else:
class Bus(object):
"""Process state-machine and messenger for HTTP site deployment.
All listeners for a given channel are guaranteed to be called even
@@ -266,14 +274,14 @@ class Bus(object):
# signal handler, console handler, or atexit handler), so we
# can't just let exceptions propagate out unhandled.
# Assume it's been logged and just die.
os._exit(70) # EX_SOFTWARE
os._exit(70) # EX_SOFTWARE
if exitstate == states.STARTING:
# exit() was called before start() finished, possibly due to
# Ctrl-C because a start listener got stuck. In this case,
# we could get stuck in a loop where Ctrl-C never exits the
# process, so we just call os.exit here.
os._exit(70) # EX_SOFTWARE
os._exit(70) # EX_SOFTWARE
def restart(self):
"""Restart the process (may close connections).
@@ -311,13 +319,21 @@ class Bus(object):
raise
# Waiting for ALL child threads to finish is necessary on OS X.
# See http://www.cherrypy.org/ticket/581.
# See https://bitbucket.org/cherrypy/cherrypy/issue/581.
# It's also good to let them all shut down before allowing
# the main thread to call atexit handlers.
# See http://www.cherrypy.org/ticket/751.
# See https://bitbucket.org/cherrypy/cherrypy/issue/751.
self.log("Waiting for child threads to terminate...")
for t in threading.enumerate():
if t != threading.currentThread() and t.isAlive():
# Validate the we're not trying to join the MainThread
# that will cause a deadlock and the case exist when
# implemented as a windows service and in any other case
# that another thread executes cherrypy.engine.exit()
if (
t != threading.currentThread() and
t.isAlive() and
not isinstance(t, threading._MainThread)
):
# Note that any dummy (external) threads are always daemonic.
if hasattr(threading.Thread, "daemon"):
# Python 2.6+
@@ -389,7 +405,7 @@ class Bus(object):
Set self.max_cloexec_files to 0 to disable this behavior.
"""
for fd in range(3, self.max_cloexec_files): # skip stdin/out/err
for fd in range(3, self.max_cloexec_files): # skip stdin/out/err
try:
flags = fcntl.fcntl(fd, fcntl.F_GETFD)
except IOError:

8
lib/cherrypy/scaffold/__init__.py
View File

@@ -47,11 +47,11 @@ Or, just look at the pretty picture:<br />
other.exposed = True
files = cherrypy.tools.staticdir.handler(
section="/files",
dir=os.path.join(local_dir, "static"),
# Ignore .php files, etc.
section="/files",
dir=os.path.join(local_dir, "static"),
# Ignore .php files, etc.
match=r'\.(css|gif|html?|ico|jpe?g|js|png|swf|xml)$',
)
)
root = Root()

13
lib/cherrypy/wsgiserver/ssl_builtin.py
View File

@@ -25,6 +25,7 @@ from cherrypy import wsgiserver
class BuiltinSSLAdapter(wsgiserver.SSLAdapter):
"""A wrapper for integrating Python's builtin ssl module with CherryPy."""
certificate = None
@@ -48,8 +49,9 @@ class BuiltinSSLAdapter(wsgiserver.SSLAdapter):
"""Wrap and return the given socket, plus WSGI environ entries."""
try:
s = ssl.wrap_socket(sock, do_handshake_on_connect=True,
server_side=True, certfile=self.certificate,
keyfile=self.private_key, ssl_version=ssl.PROTOCOL_SSLv23)
server_side=True, certfile=self.certificate,
keyfile=self.private_key,
ssl_version=ssl.PROTOCOL_SSLv23)
except ssl.SSLError:
e = sys.exc_info()[1]
if e.errno == ssl.SSL_ERROR_EOF:
@@ -77,9 +79,9 @@ class BuiltinSSLAdapter(wsgiserver.SSLAdapter):
"HTTPS": "on",
'SSL_PROTOCOL': cipher[1],
'SSL_CIPHER': cipher[0]
## SSL_VERSION_INTERFACE string The mod_ssl program version
## SSL_VERSION_LIBRARY string The OpenSSL program version
}
# SSL_VERSION_INTERFACE string The mod_ssl program version
# SSL_VERSION_LIBRARY string The OpenSSL program version
}
return ssl_environ
if sys.version_info >= (3, 0):
@@ -88,4 +90,3 @@ class BuiltinSSLAdapter(wsgiserver.SSLAdapter):
else:
def makefile(self, sock, mode='r', bufsize=DEFAULT_BUFFER_SIZE):
return wsgiserver.CP_fileobject(sock, mode, bufsize)

35
lib/cherrypy/wsgiserver/ssl_pyopenssl.py
View File

@@ -1,7 +1,7 @@
"""A library for integrating pyOpenSSL with CherryPy.
The OpenSSL module must be importable for SSL functionality.
You can obtain it from http://pyopenssl.sourceforge.net/
You can obtain it from `here <https://launchpad.net/pyopenssl>`_.
To use this module, set CherryPyWSGIServer.ssl_adapter to an instance of
SSLAdapter. There are two ways to use SSL:
@@ -44,6 +44,7 @@ except ImportError:
class SSL_fileobject(wsgiserver.CP_fileobject):
"""SSL file object attached to a socket object."""
ssl_timeout = 3
@@ -96,15 +97,8 @@ class SSL_fileobject(wsgiserver.CP_fileobject):
if time.time() - start > self.ssl_timeout:
raise socket.timeout("timed out")
def recv(self, *args, **kwargs):
buf = []
r = super(SSL_fileobject, self).recv
while True:
data = self._safe_call(True, r, *args, **kwargs)
buf.append(data)
p = self._sock.pending()
if not p:
return "".join(buf)
def recv(self, size):
return self._safe_call(True, super(SSL_fileobject, self).recv, size)
def sendall(self, *args, **kwargs):
return self._safe_call(False, super(SSL_fileobject, self).sendall,
@@ -116,6 +110,7 @@ class SSL_fileobject(wsgiserver.CP_fileobject):
class SSLConnection:
"""A thread-safe wrapper for an SSL.Connection.
``*args``: the arguments to create the wrapped ``SSL.Connection(*args)``.
@@ -151,6 +146,7 @@ class SSLConnection:
class pyOpenSSLAdapter(wsgiserver.SSLAdapter):
"""A wrapper for integrating pyOpenSSL with CherryPy."""
context = None
@@ -205,11 +201,11 @@ class pyOpenSSLAdapter(wsgiserver.SSLAdapter):
ssl_environ = {
"HTTPS": "on",
# pyOpenSSL doesn't provide access to any of these AFAICT
## 'SSL_PROTOCOL': 'SSLv2',
## SSL_CIPHER string The cipher specification name
## SSL_VERSION_INTERFACE string The mod_ssl program version
## SSL_VERSION_LIBRARY string The OpenSSL program version
}
# 'SSL_PROTOCOL': 'SSLv2',
# SSL_CIPHER string The cipher specification name
# SSL_VERSION_INTERFACE string The mod_ssl program version
# SSL_VERSION_LIBRARY string The OpenSSL program version
}
if self.certificate:
# Server certificate attributes
@@ -218,9 +214,11 @@ class pyOpenSSLAdapter(wsgiserver.SSLAdapter):
ssl_environ.update({
'SSL_SERVER_M_VERSION': cert.get_version(),
'SSL_SERVER_M_SERIAL': cert.get_serial_number(),
## 'SSL_SERVER_V_START': Validity of server's certificate (start time),
## 'SSL_SERVER_V_END': Validity of server's certificate (end time),
})
# 'SSL_SERVER_V_START':
# Validity of server's certificate (start time),
# 'SSL_SERVER_V_END':
# Validity of server's certificate (end time),
})
for prefix, dn in [("I", cert.get_issuer()),
("S", cert.get_subject())]:
@@ -253,4 +251,3 @@ class pyOpenSSLAdapter(wsgiserver.SSLAdapter):
return f
else:
return wsgiserver.CP_fileobject(sock, mode, bufsize)

435
lib/cherrypy/wsgiserver/wsgiserver2.py
View File

File diff suppressed because it is too large Load Diff

392
lib/cherrypy/wsgiserver/wsgiserver3.py
View File

@@ -86,9 +86,12 @@ import re
import email.utils
import socket
import sys
if 'win' in sys.platform and not hasattr(socket, 'IPPROTO_IPV6'):
socket.IPPROTO_IPV6 = 41
if sys.version_info < (3,1):
if 'win' in sys.platform and hasattr(socket, "AF_INET6"):
if not hasattr(socket, 'IPPROTO_IPV6'):
socket.IPPROTO_IPV6 = 41
if not hasattr(socket, 'IPV6_V6ONLY'):
socket.IPV6_V6ONLY = 27
if sys.version_info < (3, 1):
import io
else:
import _pyio as io
@@ -97,25 +100,27 @@ DEFAULT_BUFFER_SIZE = io.DEFAULT_BUFFER_SIZE
import threading
import time
from traceback import format_exc
from urllib.parse import unquote
from urllib.parse import urlparse
from urllib.parse import scheme_chars
import warnings
if sys.version_info >= (3, 0):
bytestr = bytes
unicodestr = str
basestring = (bytes, str)
def ntob(n, encoding='ISO-8859-1'):
"""Return the given native string as a byte string in the given encoding."""
"""Return the given native string as a byte string in the given
encoding.
"""
# In Python 3, the native string type is unicode
return n.encode(encoding)
else:
bytestr = str
unicodestr = unicode
basestring = basestring
def ntob(n, encoding='ISO-8859-1'):
"""Return the given native string as a byte string in the given encoding."""
"""Return the given native string as a byte string in the given
encoding.
"""
# In Python 2, the native string type is bytes. Assume it's already
# in the given encoding, which for ISO-8859-1 is almost always what
# was intended.
@@ -136,6 +141,7 @@ quoted_slash = re.compile(ntob("(?i)%2F"))
import errno
def plat_specific_errors(*errnames):
"""Return error numbers for all errors in errnames on this platform.
@@ -160,24 +166,27 @@ socket_errors_to_ignore = plat_specific_errors(
"ECONNABORTED", "WSAECONNABORTED",
"ENETRESET", "WSAENETRESET",
"EHOSTDOWN", "EHOSTUNREACH",
)
)
socket_errors_to_ignore.append("timed out")
socket_errors_to_ignore.append("The read operation timed out")
socket_errors_nonblocking = plat_specific_errors(
'EAGAIN', 'EWOULDBLOCK', 'WSAEWOULDBLOCK')
comma_separated_headers = [ntob(h) for h in
comma_separated_headers = [
ntob(h) for h in
['Accept', 'Accept-Charset', 'Accept-Encoding',
'Accept-Language', 'Accept-Ranges', 'Allow', 'Cache-Control',
'Connection', 'Content-Encoding', 'Content-Language', 'Expect',
'If-Match', 'If-None-Match', 'Pragma', 'Proxy-Authenticate', 'TE',
'Trailer', 'Transfer-Encoding', 'Upgrade', 'Vary', 'Via', 'Warning',
'WWW-Authenticate']]
'WWW-Authenticate']
]
import logging
if not hasattr(logging, 'statistics'): logging.statistics = {}
if not hasattr(logging, 'statistics'):
logging.statistics = {}
def read_headers(rfile, hdict=None):
@@ -232,7 +241,9 @@ def read_headers(rfile, hdict=None):
class MaxSizeExceeded(Exception):
pass
class SizeCheckWrapper(object):
"""Wraps a file-like object, raising MaxSizeExceeded if too large."""
def __init__(self, rfile, maxlen):
@@ -265,8 +276,8 @@ class SizeCheckWrapper(object):
self.bytes_read += len(data)
self._check_length()
res.append(data)
# See http://www.cherrypy.org/ticket/421
if len(data) < 256 or data[-1:] == "\n":
# See https://bitbucket.org/cherrypy/cherrypy/issue/421
if len(data) < 256 or data[-1:] == LF:
return EMPTY.join(res)
def readlines(self, sizehint=0):
@@ -302,6 +313,7 @@ class SizeCheckWrapper(object):
class KnownLengthRFile(object):
"""Wraps a file-like object, returning an empty string when exhausted."""
def __init__(self, rfile, content_length):
@@ -358,6 +370,7 @@ class KnownLengthRFile(object):
class ChunkedRFile(object):
"""Wraps a file-like object, returning an empty string when exhausted.
This class is intended to provide a conforming wsgi.input value for
@@ -407,8 +420,8 @@ class ChunkedRFile(object):
crlf = self.rfile.read(2)
if crlf != CRLF:
raise ValueError(
"Bad chunked transfer coding (expected '\\r\\n', "
"got " + repr(crlf) + ")")
"Bad chunked transfer coding (expected '\\r\\n', "
"got " + repr(crlf) + ")")
def read(self, size=None):
data = EMPTY
@@ -510,6 +523,7 @@ class ChunkedRFile(object):
class HTTPRequest(object):
"""An HTTP Request (and response).
A single HTTP connection may consist of multiple request/response pairs.
@@ -543,7 +557,7 @@ class HTTPRequest(object):
This value is set automatically inside send_headers."""
def __init__(self, server, conn):
self.server= server
self.server = server
self.conn = conn
self.ready = False
@@ -569,7 +583,8 @@ class HTTPRequest(object):
try:
success = self.read_request_line()
except MaxSizeExceeded:
self.simple_response("414 Request-URI Too Long",
self.simple_response(
"414 Request-URI Too Long",
"The Request-URI sent with the request exceeds the maximum "
"allowed bytes.")
return
@@ -580,7 +595,8 @@ class HTTPRequest(object):
try:
success = self.read_request_headers()
except MaxSizeExceeded:
self.simple_response("413 Request Entity Too Large",
self.simple_response(
"413 Request Entity Too Large",
"The headers sent with the request exceed the maximum "
"allowed bytes.")
return
@@ -616,12 +632,14 @@ class HTTPRequest(object):
return False
if not request_line.endswith(CRLF):
self.simple_response("400 Bad Request", "HTTP requires CRLF terminators")
self.simple_response(
"400 Bad Request", "HTTP requires CRLF terminators")
return False
try:
method, uri, req_protocol = request_line.strip().split(SPACE, 2)
# The [x:y] slicing is necessary for byte strings to avoid getting ord's
# The [x:y] slicing is necessary for byte strings to avoid getting
# ord's
rp = int(req_protocol[5:6]), int(req_protocol[7:8])
except ValueError:
self.simple_response("400 Bad Request", "Malformed Request-Line")
@@ -676,7 +694,8 @@ class HTTPRequest(object):
# Notice that, in (b), the response will be "HTTP/1.1" even though
# the client only understands 1.0. RFC 2616 10.5.6 says we should
# only return 505 if the _major_ version is different.
# The [x:y] slicing is necessary for byte strings to avoid getting ord's
# The [x:y] slicing is necessary for byte strings to avoid getting
# ord's
sp = int(self.server.protocol[5:6]), int(self.server.protocol[7:8])
if sp[0] != rp[0]:
@@ -700,7 +719,8 @@ class HTTPRequest(object):
mrbs = self.server.max_request_body_size
if mrbs and int(self.inheaders.get(b"Content-Length", 0)) > mrbs:
self.simple_response("413 Request Entity Too Large",
self.simple_response(
"413 Request Entity Too Large",
"The entity sent with the request exceeds the maximum "
"allowed bytes.")
return False
@@ -754,8 +774,10 @@ class HTTPRequest(object):
# but it seems like it would be a big slowdown for such a rare case.
if self.inheaders.get(b"Expect", b"") == b"100-continue":
# Don't use simple_response here, because it emits headers
# we don't want. See http://www.cherrypy.org/ticket/951
msg = self.server.protocol.encode('ascii') + b" 100 Continue\r\n\r\n"
# we don't want. See
# https://bitbucket.org/cherrypy/cherrypy/issue/951
msg = self.server.protocol.encode(
'ascii') + b" 100 Continue\r\n\r\n"
try:
self.conn.wfile.write(msg)
except socket.error:
@@ -791,9 +813,10 @@ class HTTPRequest(object):
if sep and QUESTION_MARK not in scheme:
# An absoluteURI.
# If there's a scheme (and it must be http or https), then:
# http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]]
# http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query
# ]]
authority, path_a, path_b = remainder.partition(FORWARD_SLASH)
return scheme.lower(), authority, path_a+path_b
return scheme.lower(), authority, path_a + path_b
if uri.startswith(FORWARD_SLASH):
# An abs_path.
@@ -823,9 +846,10 @@ class HTTPRequest(object):
cl = int(self.inheaders.get(b"Content-Length", 0))
if mrbs and mrbs < cl:
if not self.sent_headers:
self.simple_response("413 Request Entity Too Large",
"The entity sent with the request exceeds the maximum "
"allowed bytes.")
self.simple_response(
"413 Request Entity Too Large",
"The entity sent with the request exceeds the "
"maximum allowed bytes.")
return
self.rfile = KnownLengthRFile(self.conn.rfile, cl)
@@ -898,7 +922,7 @@ class HTTPRequest(object):
pass
else:
if (self.response_protocol == 'HTTP/1.1'
and self.method != b'HEAD'):
and self.method != b'HEAD'):
# Use the chunked transfer-coding
self.chunked_write = True
self.outheaders.append((b"Transfer-Encoding", b"chunked"))
@@ -934,14 +958,17 @@ class HTTPRequest(object):
self.rfile.read(remaining)
if b"date" not in hkeys:
self.outheaders.append(
(b"Date", email.utils.formatdate(usegmt=True).encode('ISO-8859-1')))
self.outheaders.append((
b"Date",
email.utils.formatdate(usegmt=True).encode('ISO-8859-1')
))
if b"server" not in hkeys:
self.outheaders.append(
(b"Server", self.server.server_name.encode('ISO-8859-1')))
buf = [self.server.protocol.encode('ascii') + SPACE + self.status + CRLF]
buf = [self.server.protocol.encode(
'ascii') + SPACE + self.status + CRLF]
for k, v in self.outheaders:
buf.append(k + COLON + SPACE + v + CRLF)
buf.append(CRLF)
@@ -949,16 +976,19 @@ class HTTPRequest(object):
class NoSSLError(Exception):
"""Exception raised when a client speaks HTTP to an HTTPS socket."""
pass
class FatalSSLAlert(Exception):
"""Exception raised when the SSL implementation signals a fatal alert."""
pass
class CP_BufferedWriter(io.BufferedWriter):
"""Faux file object attached to a socket object."""
def write(self, b):
@@ -989,7 +1019,9 @@ def CP_makefile(sock, mode='r', bufsize=DEFAULT_BUFFER_SIZE):
else:
return CP_BufferedWriter(socket.SocketIO(sock, mode), bufsize)
class HTTPConnection(object):
"""An HTTP connection (active socket).
server: the Server object which received this connection.
@@ -1040,11 +1072,14 @@ class HTTPConnection(object):
e = sys.exc_info()[1]
errnum = e.args[0]
# sadly SSL sockets return a different (longer) time out string
if errnum == 'timed out' or errnum == 'The read operation timed out':
if (
errnum == 'timed out' or
errnum == 'The read operation timed out'
):
# Don't error if we're between requests; only error
# if 1) no request has been started at all, or 2) we're
# in the middle of a request.
# See http://www.cherrypy.org/ticket/853
# See https://bitbucket.org/cherrypy/cherrypy/issue/853
if (not request_seen) or (req and req.started_request):
# Don't bother writing the 408 if the response
# has already started being written.
@@ -1072,10 +1107,12 @@ class HTTPConnection(object):
except NoSSLError:
if req and not req.sent_headers:
# Unwrap our wfile
self.wfile = CP_makefile(self.socket._sock, "wb", self.wbufsize)
req.simple_response("400 Bad Request",
"The client sent a plain HTTP request, but "
"this server only speaks HTTPS on this port.")
self.wfile = CP_makefile(
self.socket._sock, "wb", self.wbufsize)
req.simple_response(
"400 Bad Request",
"The client sent a plain HTTP request, but this server "
"only speaks HTTPS on this port.")
self.linger = True
except Exception:
e = sys.exc_info()[1]
@@ -1094,13 +1131,15 @@ class HTTPConnection(object):
self.rfile.close()
if not self.linger:
# Python's socket module does NOT call close on the kernel socket
# when you call socket.close(). We do so manually here because we
# want this server to send a FIN TCP segment immediately. Note this
# must be called *before* calling socket.close(), because the latter
# drops its reference to the kernel socket.
# Python 3 *probably* fixed this with socket._real_close; hard to tell.
## self.socket._sock.close()
# Python's socket module does NOT call close on the kernel
# socket when you call socket.close(). We do so manually here
# because we want this server to send a FIN TCP segment
# immediately. Note this must be called *before* calling
# socket.close(), because the latter drops its reference to
# the kernel socket.
# Python 3 *probably* fixed this with socket._real_close;
# hard to tell.
# self.socket._sock.close()
self.socket.close()
else:
# On the other hand, sometimes we want to hang around for a bit
@@ -1113,9 +1152,13 @@ class HTTPConnection(object):
class TrueyZero(object):
"""An object which equals and does math like the integer '0' but evals True."""
"""An object which equals and does math like the integer 0 but evals True.
"""
def __add__(self, other):
return other
def __radd__(self, other):
return other
trueyzero = TrueyZero()
@@ -1123,7 +1166,9 @@ trueyzero = TrueyZero()
_SHUTDOWNREQUEST = None
class WorkerThread(threading.Thread):
"""Thread which continuously polls a Queue for Connection objects.
Due to the timing issues of polling a Queue, a WorkerThread does not
@@ -1143,7 +1188,6 @@ class WorkerThread(threading.Thread):
"""A simple flag for the calling server to know when this thread
has begun polling the Queue."""
def __init__(self, server):
self.ready = False
self.server = server
@@ -1154,12 +1198,30 @@ class WorkerThread(threading.Thread):
self.start_time = None
self.work_time = 0
self.stats = {
'Requests': lambda s: self.requests_seen + ((self.start_time is None) and trueyzero or self.conn.requests_seen),
'Bytes Read': lambda s: self.bytes_read + ((self.start_time is None) and trueyzero or self.conn.rfile.bytes_read),
'Bytes Written': lambda s: self.bytes_written + ((self.start_time is None) and trueyzero or self.conn.wfile.bytes_written),
'Work Time': lambda s: self.work_time + ((self.start_time is None) and trueyzero or time.time() - self.start_time),
'Read Throughput': lambda s: s['Bytes Read'](s) / (s['Work Time'](s) or 1e-6),
'Write Throughput': lambda s: s['Bytes Written'](s) / (s['Work Time'](s) or 1e-6),
'Requests': lambda s: self.requests_seen + (
(self.start_time is None) and
trueyzero or
self.conn.requests_seen
),
'Bytes Read': lambda s: self.bytes_read + (
(self.start_time is None) and
trueyzero or
self.conn.rfile.bytes_read
),
'Bytes Written': lambda s: self.bytes_written + (
(self.start_time is None) and
trueyzero or
self.conn.wfile.bytes_written
),
'Work Time': lambda s: self.work_time + (
(self.start_time is None) and
trueyzero or
time.time() - self.start_time
),
'Read Throughput': lambda s: s['Bytes Read'](s) / (
s['Work Time'](s) or 1e-6),
'Write Throughput': lambda s: s['Bytes Written'](s) / (
s['Work Time'](s) or 1e-6),
}
threading.Thread.__init__(self)
@@ -1192,18 +1254,21 @@ class WorkerThread(threading.Thread):
class ThreadPool(object):
"""A Request Queue for an HTTPServer which pools threads.
ThreadPool objects must provide min, get(), put(obj), start()
and stop(timeout) attributes.
"""
def __init__(self, server, min=10, max=-1):
def __init__(self, server, min=10, max=-1,
accepted_queue_size=-1, accepted_queue_timeout=10):
self.server = server
self.min = min
self.max = max
self._threads = []
self._queue = queue.Queue()
self._queue = queue.Queue(maxsize=accepted_queue_size)
self._queue_put_timeout = accepted_queue_timeout
self.get = self._queue.get
def start(self):
@@ -1223,19 +1288,30 @@ class ThreadPool(object):
idle = property(_get_idle, doc=_get_idle.__doc__)
def put(self, obj):
self._queue.put(obj)
self._queue.put(obj, block=True, timeout=self._queue_put_timeout)
if obj is _SHUTDOWNREQUEST:
return
def grow(self, amount):
"""Spawn new worker threads (not above self.max)."""
for i in range(amount):
if self.max > 0 and len(self._threads) >= self.max:
break
worker = WorkerThread(self.server)
worker.setName("CP Server " + worker.getName())
self._threads.append(worker)
worker.start()
if self.max > 0:
budget = max(self.max - len(self._threads), 0)
else:
# self.max <= 0 indicates no maximum
budget = float('inf')
n_new = min(amount, budget)
workers = [self._spawn_worker() for i in range(n_new)]
while not all(worker.ready for worker in workers):
time.sleep(.1)
self._threads.extend(workers)
def _spawn_worker(self):
worker = WorkerThread(self.server)
worker.setName("CP Server " + worker.getName())
worker.start()
return worker
def shrink(self, amount):
"""Kill off worker threads (not below self.min)."""
@@ -1246,13 +1322,17 @@ class ThreadPool(object):
self._threads.remove(t)
amount -= 1
if amount > 0:
for i in range(min(amount, len(self._threads) - self.min)):
# Put a number of shutdown requests on the queue equal
# to 'amount'. Once each of those is processed by a worker,
# that worker will terminate and be culled from our list
# in self.put.
self._queue.put(_SHUTDOWNREQUEST)
# calculate the number of threads above the minimum
n_extra = max(len(self._threads) - self.min, 0)
# don't remove more than amount
n_to_remove = min(amount, n_extra)
# put shutdown requests on the queue equal to the number of threads
# to remove. As each request is processed by a worker, that worker
# will terminate and be culled from the list.
for n in range(n_to_remove):
self._queue.put(_SHUTDOWNREQUEST)
def stop(self, timeout=5):
# Must shut down threads here so the code that calls
@@ -1287,7 +1367,8 @@ class ThreadPool(object):
worker.join()
except (AssertionError,
# Ignore repeated Ctrl-C.
# See http://www.cherrypy.org/ticket/691.
# See
# https://bitbucket.org/cherrypy/cherrypy/issue/691.
KeyboardInterrupt):
pass
@@ -1296,12 +1377,19 @@ class ThreadPool(object):
qsize = property(_get_qsize)
try:
import fcntl
except ImportError:
try:
from ctypes import windll, WinError
import ctypes.wintypes
_SetHandleInformation = windll.kernel32.SetHandleInformation
_SetHandleInformation.argtypes = [
ctypes.wintypes.HANDLE,
ctypes.wintypes.DWORD,
ctypes.wintypes.DWORD,
]
_SetHandleInformation.restype = ctypes.wintypes.BOOL
except ImportError:
def prevent_socket_inheritance(sock):
"""Dummy function, since neither fcntl nor ctypes are available."""
@@ -1309,7 +1397,7 @@ except ImportError:
else:
def prevent_socket_inheritance(sock):
"""Mark the given socket fd as non-inheritable (Windows)."""
if not windll.kernel32.SetHandleInformation(sock.fileno(), 1, 0):
if not _SetHandleInformation(sock.fileno(), 1, 0):
raise WinError()
else:
def prevent_socket_inheritance(sock):
@@ -1320,12 +1408,14 @@ else:
class SSLAdapter(object):
"""Base class for SSL driver library adapters.
Required methods:
* ``wrap(sock) -> (wrapped socket, ssl environ dict)``
* ``makefile(sock, mode='r', bufsize=DEFAULT_BUFFER_SIZE) -> socket file object``
* ``makefile(sock, mode='r', bufsize=DEFAULT_BUFFER_SIZE) ->
socket file object``
"""
def __init__(self, certificate, private_key, certificate_chain=None):
@@ -1341,6 +1431,7 @@ class SSLAdapter(object):
class HTTPServer(object):
"""An HTTP server."""
_bind_addr = "127.0.0.1"
@@ -1353,7 +1444,8 @@ class HTTPServer(object):
"""The minimum number of worker threads to create (default 10)."""
maxthreads = None
"""The maximum number of worker threads to create (default -1 = no limit)."""
"""The maximum number of worker threads to create (default -1 = no limit).
"""
server_name = None
"""The name of the server; defaults to socket.gethostname()."""
@@ -1365,15 +1457,18 @@ class HTTPServer(object):
features used in the response."""
request_queue_size = 5
"""The 'backlog' arg to socket.listen(); max queued connections (default 5)."""
"""The 'backlog' arg to socket.listen(); max queued connections
(default 5).
"""
shutdown_timeout = 5
"""The total time, in seconds, to wait for worker threads to cleanly exit."""
"""The total time, in seconds, to wait for worker threads to cleanly exit.
"""
timeout = 10
"""The timeout in seconds for accepted connections (default 10)."""
version = "CherryPy/3.2.2"
version = "CherryPy/3.6.0"
"""A version string for the HTTPServer."""
software = None
@@ -1382,7 +1477,9 @@ class HTTPServer(object):
If None, this defaults to ``'%s Server' % self.version``."""
ready = False
"""An internal flag which marks whether the socket is accepting connections."""
"""An internal flag which marks whether the socket is accepting
connections.
"""
max_request_header_size = 0
"""The maximum size, in bytes, for request headers, or 0 for no limit."""
@@ -1426,14 +1523,15 @@ class HTTPServer(object):
'Threads': lambda s: len(getattr(self.requests, "_threads", [])),
'Threads Idle': lambda s: getattr(self.requests, "idle", None),
'Socket Errors': 0,
'Requests': lambda s: (not s['Enabled']) and -1 or sum([w['Requests'](w) for w
in s['Worker Threads'].values()], 0),
'Bytes Read': lambda s: (not s['Enabled']) and -1 or sum([w['Bytes Read'](w) for w
in s['Worker Threads'].values()], 0),
'Bytes Written': lambda s: (not s['Enabled']) and -1 or sum([w['Bytes Written'](w) for w
in s['Worker Threads'].values()], 0),
'Work Time': lambda s: (not s['Enabled']) and -1 or sum([w['Work Time'](w) for w
in s['Worker Threads'].values()], 0),
'Requests': lambda s: (not s['Enabled']) and -1 or sum(
[w['Requests'](w) for w in s['Worker Threads'].values()], 0),
'Bytes Read': lambda s: (not s['Enabled']) and -1 or sum(
[w['Bytes Read'](w) for w in s['Worker Threads'].values()], 0),
'Bytes Written': lambda s: (not s['Enabled']) and -1 or sum(
[w['Bytes Written'](w) for w in s['Worker Threads'].values()],
0),
'Work Time': lambda s: (not s['Enabled']) and -1 or sum(
[w['Work Time'](w) for w in s['Worker Threads'].values()], 0),
'Read Throughput': lambda s: (not s['Enabled']) and -1 or sum(
[w['Bytes Read'](w) / (w['Work Time'](w) or 1e-6)
for w in s['Worker Threads'].values()], 0),
@@ -1441,7 +1539,7 @@ class HTTPServer(object):
[w['Bytes Written'](w) / (w['Work Time'](w) or 1e-6)
for w in s['Worker Threads'].values()], 0),
'Worker Threads': {},
}
}
logging.statistics["CherryPy HTTPServer %d" % id(self)] = self.stats
def runtime(self):
@@ -1456,6 +1554,7 @@ class HTTPServer(object):
def _get_bind_addr(self):
return self._bind_addr
def _set_bind_addr(self, value):
if isinstance(value, tuple) and value[0] in ('', None):
# Despite the socket module docs, using '' does not
@@ -1472,7 +1571,9 @@ class HTTPServer(object):
"Use '0.0.0.0' (IPv4) or '::' (IPv6) instead "
"to listen on all active interfaces.")
self._bind_addr = value
bind_addr = property(_get_bind_addr, _set_bind_addr,
bind_addr = property(
_get_bind_addr,
_set_bind_addr,
doc="""The interface on which to listen for connections.
For TCP sockets, a (host, port) tuple. Host values may be any IPv4
@@ -1500,21 +1601,28 @@ class HTTPServer(object):
# AF_UNIX socket
# So we can reuse the socket...
try: os.unlink(self.bind_addr)
except: pass
try:
os.unlink(self.bind_addr)
except:
pass
# So everyone can access the socket...
try: os.chmod(self.bind_addr, 511) # 0777
except: pass
try:
os.chmod(self.bind_addr, 511) # 0777
except:
pass
info = [(socket.AF_UNIX, socket.SOCK_STREAM, 0, "", self.bind_addr)]
info = [
(socket.AF_UNIX, socket.SOCK_STREAM, 0, "", self.bind_addr)]
else:
# AF_INET or AF_INET6 socket
# Get the correct address family for our host (allows IPv6 addresses)
# Get the correct address family for our host (allows IPv6
# addresses)
host, port = self.bind_addr
try:
info = socket.getaddrinfo(host, port, socket.AF_UNSPEC,
socket.SOCK_STREAM, 0, socket.AI_PASSIVE)
socket.SOCK_STREAM, 0,
socket.AI_PASSIVE)
except socket.gaierror:
if ':' in self.bind_addr[0]:
info = [(socket.AF_INET6, socket.SOCK_STREAM,
@@ -1529,7 +1637,8 @@ class HTTPServer(object):
af, socktype, proto, canonname, sa = res
try:
self.bind(af, socktype, proto)
except socket.error:
except socket.error as serr:
msg = "%s -- (%s: %s)" % (msg, sa, serr)
if self.socket:
self.socket.close()
self.socket = None
@@ -1583,11 +1692,13 @@ class HTTPServer(object):
self.socket = self.ssl_adapter.bind(self.socket)
# If listening on the IPV6 any address ('::' = IN6ADDR_ANY),
# activate dual-stack. See http://www.cherrypy.org/ticket/871.
# activate dual-stack. See
# https://bitbucket.org/cherrypy/cherrypy/issue/871.
if (hasattr(socket, 'AF_INET6') and family == socket.AF_INET6
and self.bind_addr[0] in ('::', '::0', '::0.0.0.0')):
and self.bind_addr[0] in ('::', '::0', '::0.0.0.0')):
try:
self.socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0)
self.socket.setsockopt(
socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0)
except (AttributeError, socket.error):
# Apparently, the socket option is not available in
# this machine's TCP stack
@@ -1642,7 +1753,7 @@ class HTTPServer(object):
if not isinstance(self.bind_addr, basestring):
# optional values
# Until we do DNS lookups, omit REMOTE_HOST
if addr is None: # sometimes this can happen
if addr is None: # sometimes this can happen
# figure out if AF_INET or AF_INET6.
if len(s.getsockname()) == 2:
# AF_INET
@@ -1655,7 +1766,12 @@ class HTTPServer(object):
conn.ssl_env = ssl_env
self.requests.put(conn)
try:
self.requests.put(conn)
except queue.Full:
# Just drop the conn. TODO: write 503 back?
conn.close()
return
except socket.timeout:
# The only reason for the timeout in start() is so we can
# notice keyboard interrupts on Win32, which don't interrupt
@@ -1670,19 +1786,22 @@ class HTTPServer(object):
# is received during the accept() call; all docs say retry
# the call, and I *think* I'm reading it right that Python
# will then go ahead and poll for and handle the signal
# elsewhere. See http://www.cherrypy.org/ticket/707.
# elsewhere. See
# https://bitbucket.org/cherrypy/cherrypy/issue/707.
return
if x.args[0] in socket_errors_nonblocking:
# Just try again. See http://www.cherrypy.org/ticket/479.
# Just try again. See
# https://bitbucket.org/cherrypy/cherrypy/issue/479.
return
if x.args[0] in socket_errors_to_ignore:
# Our socket was closed.
# See http://www.cherrypy.org/ticket/686.
# See https://bitbucket.org/cherrypy/cherrypy/issue/686.
return
raise
def _get_interrupt(self):
return self._interrupt
def _set_interrupt(self, interrupt):
self._interrupt = True
self.stop()
@@ -1708,7 +1827,8 @@ class HTTPServer(object):
x = sys.exc_info()[1]
if x.args[0] not in socket_errors_to_ignore:
# Changed to use error code and not message
# See http://www.cherrypy.org/ticket/860.
# See
# https://bitbucket.org/cherrypy/cherrypy/issue/860.
raise
else:
# Note that we're explicitly NOT using AI_PASSIVE,
@@ -1721,8 +1841,9 @@ class HTTPServer(object):
s = None
try:
s = socket.socket(af, socktype, proto)
# See http://groups.google.com/group/cherrypy-users/
# browse_frm/thread/bbfe5eb39c904fe0
# See
# http://groups.google.com/group/cherrypy-users/
# browse_frm/thread/bbfe5eb39c904fe0
s.settimeout(1.0)
s.connect((host, port))
s.close()
@@ -1737,7 +1858,9 @@ class HTTPServer(object):
class Gateway(object):
"""A base class to interface HTTPServer with other systems, such as WSGI."""
"""A base class to interface HTTPServer with other systems, such as WSGI.
"""
def __init__(self, req):
self.req = req
@@ -1751,7 +1874,8 @@ class Gateway(object):
# of such classes (in which case they will be lazily loaded).
ssl_adapters = {
'builtin': 'cherrypy.wsgiserver.ssl_builtin.BuiltinSSLAdapter',
}
}
def get_ssl_adapter_class(name='builtin'):
"""Return an SSL adapter class for the given name."""
@@ -1778,18 +1902,22 @@ def get_ssl_adapter_class(name='builtin'):
return adapter
# -------------------------------- WSGI Stuff -------------------------------- #
# ------------------------------- WSGI Stuff -------------------------------- #
class CherryPyWSGIServer(HTTPServer):
"""A subclass of HTTPServer which calls a WSGI application."""
wsgi_version = (1, 0)
"""The version of WSGI to produce."""
def __init__(self, bind_addr, wsgi_app, numthreads=10, server_name=None,
max=-1, request_queue_size=5, timeout=10, shutdown_timeout=5):
self.requests = ThreadPool(self, min=numthreads or 1, max=max)
max=-1, request_queue_size=5, timeout=10, shutdown_timeout=5,
accepted_queue_size=-1, accepted_queue_timeout=10):
self.requests = ThreadPool(self, min=numthreads or 1, max=max,
accepted_queue_size=accepted_queue_size,
accepted_queue_timeout=accepted_queue_timeout)
self.wsgi_app = wsgi_app
self.gateway = wsgi_gateways[self.wsgi_version]
@@ -1805,12 +1933,14 @@ class CherryPyWSGIServer(HTTPServer):
def _get_numthreads(self):
return self.requests.min
def _set_numthreads(self, value):
self.requests.min = value
numthreads = property(_get_numthreads, _set_numthreads)
class WSGIGateway(Gateway):
"""A base class to interface HTTPServer with WSGI."""
def __init__(self, req):
@@ -1842,7 +1972,7 @@ class WSGIGateway(Gateway):
if hasattr(response, "close"):
response.close()
def start_response(self, status, headers, exc_info = None):
def start_response(self, status, headers, exc_info=None):
"""WSGI callable to begin the HTTP response."""
# "The application may call start_response more than once,
# if and only if the exc_info argument is provided."
@@ -1870,12 +2000,15 @@ class WSGIGateway(Gateway):
for k, v in headers:
if not isinstance(k, str):
raise TypeError("WSGI response header key %r is not of type str." % k)
raise TypeError(
"WSGI response header key %r is not of type str." % k)
if not isinstance(v, str):
raise TypeError("WSGI response header value %r is not of type str." % v)
raise TypeError(
"WSGI response header value %r is not of type str." % v)
if k.lower() == 'content-length':
self.remaining_bytes_out = int(v)
self.req.outheaders.append((k.encode('ISO-8859-1'), v.encode('ISO-8859-1')))
self.req.outheaders.append(
(k.encode('ISO-8859-1'), v.encode('ISO-8859-1')))
return self.write
@@ -1894,8 +2027,9 @@ class WSGIGateway(Gateway):
if not self.req.sent_headers:
# Whew. We can send a 500 to the client.
self.req.simple_response("500 Internal Server Error",
"The requested resource returned more bytes than the "
"declared Content-Length.")
"The requested resource returned "
"more bytes than the declared "
"Content-Length.")
else:
# Dang. We have probably already sent data. Truncate the chunk
# to fit (so the client doesn't hang) and raise an error later.
@@ -1915,6 +2049,7 @@ class WSGIGateway(Gateway):
class WSGIGateway_10(WSGIGateway):
"""A Gateway class to interface HTTPServer with WSGI 1.0.x."""
def get_environ(self):
@@ -1930,7 +2065,7 @@ class WSGIGateway_10(WSGIGateway):
'REMOTE_ADDR': req.conn.remote_addr or '',
'REMOTE_PORT': str(req.conn.remote_port or ''),
'REQUEST_METHOD': req.method.decode('ISO-8859-1'),
'REQUEST_URI': req.uri,
'REQUEST_URI': req.uri.decode('ISO-8859-1'),
'SCRIPT_NAME': '',
'SERVER_NAME': req.server.server_name,
# Bah. "SERVER_PROTOCOL" is actually the REQUEST protocol.
@@ -1943,8 +2078,7 @@ class WSGIGateway_10(WSGIGateway):
'wsgi.run_once': False,
'wsgi.url_scheme': req.scheme.decode('ISO-8859-1'),
'wsgi.version': (1, 0),
}
}
if isinstance(req.server.bind_addr, basestring):
# AF_UNIX. This isn't really allowed by WSGI, which doesn't
# address unix domain sockets. But it's better than nothing.
@@ -1972,10 +2106,11 @@ class WSGIGateway_10(WSGIGateway):
class WSGIGateway_u0(WSGIGateway_10):
"""A Gateway class to interface HTTPServer with WSGI u.0.
WSGI u.0 is an experimental protocol, which uses unicode for keys and values
in both Python 2 and Python 3.
WSGI u.0 is an experimental protocol, which uses unicode for keys
and values in both Python 2 and Python 3.
"""
def get_environ(self):
@@ -2004,7 +2139,9 @@ wsgi_gateways = {
('u', 0): WSGIGateway_u0,
}
class WSGIPathInfoDispatcher(object):
"""A WSGI dispatcher for dispatch based on the PATH_INFO.
apps: a dict or list of (path_prefix, app) pairs.
@@ -2037,4 +2174,3 @@ class WSGIPathInfoDispatcher(object):
start_response('404 Not Found', [('Content-Type', 'text/plain'),
('Content-Length', '0')])
return ['']