Release v0.5.7

Updated changelog for 0.5.7 release
Updated changelog to reflect metacritic changes
2026-09-09 16:22:52 +01:00 · 2015-07-01 02:18:01 -07:00 · 2015-07-01 02:17:51 -07:00 · 2015-07-01 02:08:13 -07:00 · 2015-07-01 02:06:27 -07:00 · 2015-07-01 00:56:48 -07:00
937 changed files with 43109 additions and 14554 deletions
@@ -6,12 +6,13 @@
 *.pyproj
 *.sln

-# Logs and databases #
+# Headphones files #
 ######################
 *.log
 *.db*
 *.db-journal
 *.ini
+version.lock
 logs/*
 cache/*

@@ -19,6 +20,7 @@ cache/*
 ##################
 *.crt
 *.key
+*.csr

 # OS generated files #
 ######################
@@ -0,0 +1,17 @@
+[pep8]
+# E111 indentation is not a multiple of four
+# E121 continuation line under-indented for hanging indent
+# E122 continuation line missing indentation or outdented
+# E124 closing bracket does not match visual indentation
+# E125 continuation line with same indent as next logical line
+# E126 continuation line over-indented for hanging indent
+# E127 continuation line over-indented for visual indent
+# E128 continuation line under-indented for visual indent
+# E261 at least two spaces before inline comment
+# E262 inline comment should start with '# '
+# E265 block comment should start with '# '
+# E302 expected 2 blank lines, found 1
+# E501 line too long (312 > 160 characters)
+# E502 the backslash is redundant between brackets
+ignore = E111,E121,E122,E123,E124,E125,E126,E127,E128,E261,E262,E265,E302,E501,E502
+max-line-length = 160
@@ -0,0 +1,21 @@
+# Travis CI configuration file
+# http://about.travis-ci.org/docs/
+
+language: python
+
+# Available Python versions:
+# http://about.travis-ci.org/docs/user/ci-environment/#Python-VM-images
+python:
+  - "2.6"
+  - "2.7"
+# pylint 1.4 does not run under python 2.6
+install:
+ - pip install pyOpenSSL
+ - pip install pylint==1.3.1
+ - pip install pyflakes
+ - pip install pep8
+script:
+  - pep8 headphones
+  - pylint --rcfile=pylintrc headphones
+  - pyflakes headphones
+  - nosetests headphones
@@ -0,0 +1,111 @@
+# API Reference
+The API is still pretty new and needs some serious cleaning up on the backend but should be reasonably functional. There are no error codes yet.
+
+## General structure
+The API endpoint is `http://ip:port + HTTP_ROOT + /api?apikey=$apikey&cmd=$command`
+
+Data response in JSON formatted. If executing a command like "delArtist" or "addArtist" you'll get back an "OK", else, you'll get the data you requested.
+
+## API methods
+
+### getIndex
+Fetch data from index page. Returns: ArtistName, ArtistSortName, ArtistID, Status, DateAdded, [LatestAlbum, ReleaseDate, AlbumID], HaveTracks, TotalTracks, IncludeExtras, LastUpdated, [ArtworkURL, ThumbURL]: a remote url to the artwork/thumbnail.
+
+To get the cached image path, see getArtistArt command. ThumbURL is added/updated when an artist is added/updated. If your using the database method to get the artwork, it's more reliable to use the ThumbURL than the ArtworkURL)
+
+### getArtist&id=$artistid
+Fetch artist data. returns the artist object (see above) and album info: Status, AlbumASIN, DateAdded, AlbumTitle, ArtistName, ReleaseDate, AlbumID, ArtistID, Type, ArtworkURL: hosted image path. For cached image, see getAlbumArt command)
+
+### getAlbum&id=$albumid
+Fetch data from album page. Returns the album object, a description object and a tracks object. Tracks contain: AlbumASIN, AlbumTitle, TrackID, Format, TrackDuration (ms), ArtistName, TrackTitle, AlbumID, ArtistID, Location, TrackNumber, CleanName (stripped of punctuation /styling), BitRate)
+
+### getUpcoming
+Returns: Status, AlbumASIN, DateAdded, AlbumTitle, ArtistName, ReleaseDate, AlbumID, ArtistID, Type
+
+### getWanted
+Returns: Status, AlbumASIN, DateAdded, AlbumTitle, ArtistName, ReleaseDate, AlbumID, ArtistID, Type
+
+### getSimilar
+Returns similar artists  - with a higher "Count" being more likely to be similar. Returns: Count, ArtistName, ArtistID
+
+### getHistory
+Returns: Status, DateAdded, Title, URL (nzb), FolderName, AlbumID, Size (bytes)
+
+### getLogs
+Not working yet
+
+### findArtist&name=$artistname[&limit=$limit]
+Perform artist query on musicbrainz. Returns: url, score, name, uniquename (contains disambiguation info), id)
+
+### findAlbum&name=$albumname[&limit=$limit]
+Perform album query on musicbrainz. Returns: title, url (artist), id (artist), albumurl, albumid, score, uniquename (artist - with disambiguation)
+
+### addArtist&id=$artistid
+Add an artist to the db by artistid)
+
+### addAlbum&id=$releaseid
+Add an album to the db by album release id
+
+### delArtist&id=$artistid
+Delete artist from db by artistid)
+
+### pauseArtist&id=$artistid
+Pause an artist in db)
+
+### resumeArtist&id=$artistid
+Resume an artist in db)
+
+### refreshArtist&id=$artistid
+Refresh info for artist in db from musicbrainz
+
+### queueAlbum&id=$albumid[&new=True&lossless=True]
+Mark an album as wanted and start the searcher. Optional paramters: 'new' looks for new versions, 'lossless' looks only for lossless versions
+
+### unqueueAlbum&id=$albumid
+Unmark album as wanted / i.e. mark as skipped
+
+### forceSearch
+force search for wanted albums - not launched in a separate thread so it may take a bit to complete
+### forceProcess
+Force post process albums in download directory - also not launched in a separate thread
+### forceActiveArtistsUpdate
+force Active Artist Update - also not launched in a separate thread
+
+### getVersion
+Returns some version information: git_path, install_type, current_version, installed_version, commits_behind
+
+### checkGithub
+Updates the version information above and returns getVersion data
+
+### shutdown
+Shut down headphones
+
+### restart
+Restart headphones
+
+### update
+Update headphones - you may want to check the install type in get version and not allow this if type==exe
+
+### getArtistArt&id=$artistid
+Returns either a relative path to the cached image, or a remote url if the image can't be saved to the cache dir
+
+getAlbumArt&id=$albumid
+see above
+
+### getArtistInfo&id=$artistid
+Returns Summary and Content, both formatted in html.
+
+### getAlbumInfo&id=$albumid
+See above, returns Summary and Content.
+
+### getArtistThumb&id=$artistid
+Returns either a relative path to the cached thumbnail artist image, or an http:// address if the cache dir can't be written to.
+
+### getAlbumThumb&id=$albumid
+See above.
+
+### choose_specific_download&id=$albumid
+Gives you a list of results from searcher.searchforalbum(). Basically runs a normal search, but rather than sorting them and downloading the best result, it dumps the data, which you can then pass on to download_specific_release(). Returns a list of dictionaries with params: title, size, url, provider & kind - all of these values must be passed back to download_specific_release
+
+### download_specific_release&id=albumid&title=$title&size=$size&url=$url&provider=$provider&kind=$kind
+Allows you to manually pass a choose_specific_download release back to searcher.send_to_downloader()
@@ -1,71 +0,0 @@
-The API is still pretty new and needs some serious cleaning up on the backend but should be
-reasonably functional. There are no error codes yet, 
-
-
-General structure:
-http://localhost:8181 + HTTP_ROOT + /api?apikey=$apikey&cmd=$command
-
-Data returned in json format. If executing a command like "delArtist" or "addArtist" you'll get back an "OK", else, you'll get the data you requested
-
-$commands&parameters[&optionalparameters]:
-
-
-getIndex (fetch data from index page. Returns: ArtistName, ArtistSortName, ArtistID, Status, DateAdded,
-												[LatestAlbum, ReleaseDate, AlbumID], HaveTracks, TotalTracks,
-												IncludeExtras, LastUpdated, [ArtworkURL, ThumbURL]: a remote url to the artwork/thumbnail. To get the cached image path, see getArtistArt command.
-                                                                                                    ThumbURL is added/updated when an artist is added/updated. If your using the database method to get the artwork,
-                                                                                                    it's more reliable to use the ThumbURL than the ArtworkURL)
-
-getArtist&id=$artistid (fetch artist data. returns the artist object (see above) and album info: Status, AlbumASIN, DateAdded, AlbumTitle, ArtistName, ReleaseDate, AlbumID, ArtistID, Type, ArtworkURL: hosted image path. For cached image, see getAlbumArt command)
-
-getAlbum&id=$albumid (fetch data from album page. Returns the album object, a description object and a tracks object. Tracks contain: AlbumASIN, AlbumTitle, TrackID, Format, TrackDuration (ms), ArtistName, TrackTitle, AlbumID, ArtistID, Location, TrackNumber, CleanName (stripped of punctuation /styling), BitRate)
-
-getUpcoming (Returns: Status, AlbumASIN, DateAdded, AlbumTitle, ArtistName, ReleaseDate, AlbumID, ArtistID, Type)
-
-getWanted (Returns: Status, AlbumASIN, DateAdded, AlbumTitle, ArtistName, ReleaseDate, AlbumID, ArtistID, Type)
-
-getSimilar (Returns similar artists  - with a higher "Count" being more likely to be similar. Returns: Count, ArtistName, ArtistID)
-
-getHistory (Returns: Status, DateAdded, Title, URL (nzb), FolderName, AlbumID, Size (bytes))
-
-getLogs (not working yet)
-
-findArtist&name=$artistname[&limit=$limit] (perform artist query on musicbrainz. Returns: url, score, name, uniquename (contains disambiguation info), id)
-
-findAlbum&name=$albumname[&limit=$limit]  (perform album query on musicbrainz. Returns: title, url (artist), id (artist), albumurl, albumid, score, uniquename (artist - with disambiguation)
-
-addArtist&id=$artistid (add an artist to the db by artistid)
-addAlbum&id=$releaseid (add an album to the db by album release id)
-
-delArtist&id=$artistid (delete artist from db by artistid)
-
-pauseArtist&id=$artistid (pause an artist in db)
-resumeArtist&id=$artistid (resume an artist in db)
-
-refreshArtist&id=$artistid (refresh info for artist in db from musicbrainz)
-
-queueAlbum&id=$albumid[&new=True&lossless=True] (Mark an album as wanted and start the searcher. Optional paramters: 'new' looks for new versions, 'lossless' looks only for lossless versions)
-unqueueAlbum&id=$albumid (Unmark album as wanted / i.e. mark as skipped)
-
-forceSearch (force search for wanted albums - not launched in a separate thread so it may take a bit to complete)
-forceProcess (force post process albums in download directory - also not launched in a separate thread)
-
-getVersion (Returns some version information: git_path, install_type, current_version, installed_version, commits_behind
-checkGithub (updates the version information above and returns getVersion data)
-
-shutdown (shut down headphones)
-restart (restart headphones)
-update (update headphones - you may want to check the install type in get version and not allow this if type==exe)
-
-getArtistArt&id=$artistid  (Returns either a relative path to the cached image, or a remote url if the image can't be saved to the cache dir)
-getAlbumArt&id=$albumid (see above)
-
-getArtistInfo&id=$artistid (Returns Summary and Content, both formatted in html)
-getAlbumInfo&id=$albumid (See above, returns Summary and Content)
-
-getArtistThumb&id=$artistid (Returns either a relative path to the cached thumbnail artist image, or an http:// address if the cache dir can't be written to)
-getAlbumThumb&id=$albumid (see above)
-
-choose_specific_download&id=$albumid (Gives you a list of results from searcher.searchforalbum(). Basically runs a normal search, but rather than sorting them and downloading the best result, it dumps the data, which you can then pass on to download_specific_release(). Returns a list of dictionaries with params: title, size, url, provider & kind - all of these values must be passed back to download_specific_release)
-
-download_specific_release&id=albumid&title=$title&size=$size&url=$url&provider=$provider&kind=$kind (Allows you to manually pass a choose_specific_download release back to searcher.send_to_downloader())
@@ -0,0 +1,212 @@
+# Changelog
+
+## v0.5.7
+Released 01 July 2015
+
+Highlights:
+* Improved: Moved pushover to use requests lib
+* Improved: Plex tokens with Plex Home
+* Improved: Added getLogs & clearLogs to api
+* Improved: Cache MetaCritic scores. Added user-agent header to fix 403 errors
+* Improved: Specify whether to delete folders when force post-processing
+* Improved: Convert target bitrate to vbr preset for what.cd searching
+* Improved: Switched Pushover to requests lib
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.5.6...v0.5.7).
+
+## v0.5.6
+Released 08 June 2015
+
+Highlights:
+* Added: Metacritic scores
+* Added: Series support (e.g. Cafe Del Mar, Now That's What I Call Music, etc)
+* Added: Filter out clean/edited/censored releases (#2198)
+* Added: Button on the log page to toggle verbose/debug logging
+* Fixed: Connecting to SABnzbd over https with python >= 2.7.9
+* Fixed: Email Notifications with SSL
+* Fixed: Don't limit musicbrainz results to first 100
+* Fixed: nzbget url fix
+* Fixed: OSX Notifications
+* Improved: Cuesplit, allow wav, ape to be split
+* Improved: Moved the 'freeze db' option to the advanced->misc. tab
+* Improved: Moved kickass searching to json api, so it doesn't throw 404 errors anymore when there are no results
+* Improved: SSL for headphones indexer
+* Improved: Disable update dialog box if check_github is diabled
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.5.5...v0.5.6).
+
+## v0.5.5
+Released 04 May 2015
+
+Highlights:
+* Added: force ID3v2.3 during post processing (#2121)
+* Added: MusicBrainz authentication (#2125)
+* Added: Email notifications (addresses #1045)
+* Fixed: Kickass url updated to kickass.to (#2119)
+* Fixed: Piratebay returning 0 bytes for all files
+* Fixed: Albums stopped automatically refreshing when adding an artist
+* Fixed: Min/max sizes for target bitrate
+* Fixed: Don't filter any results if looking for a specific download
+* Fixed: Sort by size in the specific download table
+* Fixed: Deal with beets recommendation.none correctly
+* Improved: Close dialog window automatically when choosing a specific download
+* Improved: Move some repetitive log messages to debug level
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.5.4...v0.5.5).
+
+## v0.5.4
+Released 05 February 2015
+
+Highlights:
+* Added: backported 'Scan Artist' feature from `sarakha63/headphones`
+* Fixed: change file permissions of cache files according to settings (#2102)
+* Fixed: hide Songkick if no regional events are available
+* Fixed: only reschedule jobs if timeout changed (#2099)
+* Fixed: limit dialog height (#2096)
+* Improved: upgraded requests to 2.5.1
+* Improved: upgraded unidecode to 0.04.17
+* Improved: upgraded tzlocal to 1.1.2
+* Improved: upgraded PyTZ to 2014.10
+* Improved: upgraded mako to 1.0.1
+* Improved: upgraded mutagen to 1.27
+* Improved: upgraded beets to 1.3.10
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.5.3...v0.5.4).
+
+## v0.5.3
+Released 15 January 2015
+
+Highlights:
+* Added: update active artists via API (#2075)
+* Fixed: queue instance lacks `add` method (#2055)
+* Improved: better SSL error messages (#2058)
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.5.2...v0.5.3).
+
+## v0.5.2
+Released 28 December 2014
+
+Highlights:
+* Added: advanced option to ignore certain folders by patterns. (#2037)
+* Added: advanced option to ignore certain files by patterns (library only)
+* Added: specify optional paths to CUE splitting tools (#1938)
+* Added: experimental support for OPB (provide details yourself!)
+* Fixed: magnet to torrent conversion (#1926)
+* Fixed: new KAT URL (#2043)
+* Fixed: LMS notifications (#1564)
+* Improved: notify user of SSL-related warnings, instead of silently failing.
+* Improved: show all search results for 'Choose Specific Release'
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.5.1...v0.5.2).
+
+## v0.5.1
+Released 13 November 2014
+
+Highlights:
+* Added: allow one to disable interval tasks (#2002)
+* Added: script to downgrade Headphones to last version that started (Linux only)
+* Fixed: SSL issues in CherryPy. Self-generated certificates will be 2048 now (#1995)
+* Fixed: Transmission URL detection (#1998)
+* Fixed: missing dependencies for APScheduler and CherryPy (#2001)
+* Improved: symlink infinite recursion detection
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.5...v0.5.1).
+
+## v0.5
+Released 10 November 2014
+
+Highlights:
+* Added: CUE splitter
+* Added: filter search result by MusicBrainz Release Group ID
+* Added: follow symlinks while scanning library (#1953)
+* Fixed: crash during post processing (#1897)
+* Fixed: embedding lyrics (#1896)
+* Fixed: HTTP errors with older versions of Python 2.6
+* Fixed: jump back to top of page (#1948)
+* Improved: parse MusicBrainz RGID first when post processing (#1952)
+* Improved: Growl unicode characters (#1695)
+* Improved: search handling for PB and KAT
+* Improved: Last.FM API support (#1877)
+* Improved: upgraded CherryPy to version 3.6.0
+* Improved: upgraded Requests to version 2.4.1
+* Improved: upgraded APScheduler to version 3.0.1
+* Improved: lot of code refactoring
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.4...v0.5).
+
+## v0.4
+Released 20 September 2014
+
+Highlights:
+* Added: support for libav-tools (which replaces FFmpeg under Ubuntu)
+* Added: option to freeze library when post processing
+* Added: Songkick per area
+* Added: rename original NFO file (#1797)
+* Removed: removed dead search providers
+* Fixed: removed left-overs of old packages
+* Improved: rename by original folder (#1811)
+* Improved: uTorrent fixess
+* Improved: delayed loading of album art
+* Improved: search result parser
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.3.4...v0.4).
+
+## v0.3.4
+Released 15 May 2014
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.3.3...v0.3.4).
+
+## v0.3.3
+Released 05 May 2014
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.3.2...v0.3.3).
+
+## v0.3.2
+Released 17 April 2014
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.3.1...v0.3.2).
+
+## v0.3.1
+Released 11 April 2014
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.3...v0.3.1).
+
+## v0.3
+Released 09 April 2014
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.2.3...v0.3).
+
+## v0.2.3
+Released 28 March 2014
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.2.2...v0.2.3).
+
+## v0.2.2
+Released 15 January 2014
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.2.1...v0.2.2).
+
+## v0.2.1
+Released 13 January 2014
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.2...v0.2.1).
+
+## v0.2
+Released 13 January 2014
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.1.2...v0.2).
+
+## v0.1.2
+Released 11 October 2013
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.1.1...v0.1.2).
+
+## v0.1.1
+Released 26 August 2013
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/v0.1...v0.1.1).
+
+## v0.1
+Released 05 August 2013
+
+The full list of commits can be found [here](https://github.com/rembo10/headphones/compare/2156e1341405d07c5bcfbe994f6b354b32d94cda...v0.1).
@@ -0,0 +1,52 @@
+# Contributing to Headphones
+
+## Issues
+In case you read this because you are posting an issue, please take a minute and conside the things below. The issue tracker is not a support forum. It is primarily intended to submit bugs, improvements or feature requests. However, we are glad to help you, and make sure the problem is not caused by Headphones, but don't expect step-by-step answers.
+
+* Use the search function. Chances are that your problem is already discussed. Do not append to (closed) issues if your problem does not fit the discussion.
+* Visit the [Troubleshooting](../../wiki/TroubleShooting) wiki first.
+* Use [proper formatting](https://help.github.com/articles/github-flavored-markdown/). Paste your logs in code blocks.
+* Close your issue if you resolved it.
+
+## Pull Requests
+If you think you can contribute code to the Headphones repository, do not hesitate to submit a pull request.
+
+### Branches
+All pull requests should be based on the `develop` branch, to minimize cross merges. When you want to develop a new feature, clone the repository with `git clone origin/develop -b FEATURE_NAME`. Use meaningful commit messages.
+
+### Python Code
+
+#### Compatibility
+The code should work with Python 2.6 and 2.7. Note that Headphones runs on different platforms, including Network Attached Storage devices such as Synology.
+
+Re-use existing code. Do not hesitate to add logging in your code. You can the logger module `headphones.logger.*` for this. Web requests are invoked via `headphones.request.*` and derived ones. Use these methods to automatically add proper and meaningful error handling.
+
+#### Code conventions
+Although Headphones did not adapt a code convention in the past, we try to follow the [PEP8](http://legacy.python.org/dev/peps/pep-0008/) conventions for future code. A short summary to remind you (copied from http://wiki.ros.org/PyStyleGuide):
+
+ * 4 space indentation
+ * 80 characters per line
+ * `package_name`
+ * `ClassName`
+ * `method_name`
+ * `field_name`
+ * `_private_something`
+ * `self.__really_private_field`
+ * `_global`
+
+#### Documentation
+Document your code. Use docstrings See [PEP-257](https://www.python.org/dev/peps/pep-0257/) for more information.
+
+#### Continuous Integration
+Headphones has a configuration file for [travis-ci](https://travis-ci.org/). You can add your forked repo to Travis to have it check your code against PEP8, PyLint, and PyFlakes for you. Your pull request will show a green check mark or a red cross on each tested commit, depending on if linting passes.
+
+### HTML/Template code
+
+#### Compatibility
+HTML5 compatible browsers are targetted. There is no specific mobile version of Headphones yet.
+
+#### Conventions
+* 4 space indentation
+* `methodName`
+* `variableName`
+* `ClassName`
@@ -14,15 +14,14 @@
 #  You should have received a copy of the GNU General Public License
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

-import os, sys
+import os
+import sys

 # Ensure lib added to path, before any other imports
 sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'lib/'))

 from headphones import webstart, logger

-from configobj import ConfigObj
-
 import locale
 import time
 import signal
@@ -33,6 +32,7 @@ import headphones
 signal.signal(signal.SIGINT, headphones.sig_handler)
 signal.signal(signal.SIGTERM, headphones.sig_handler)

+
 def main():
    """
    Headphones application entry point. Parses arguments, setups encoding and
@@ -63,16 +63,24 @@ def main():
        headphones.SYS_ENCODING = 'UTF-8'

    # Set up and gather command line arguments
-    parser = argparse.ArgumentParser(description='Music add-on for SABnzbd+, Transmission and more.')
+    parser = argparse.ArgumentParser(
+        description='Music add-on for SABnzbd+, Transmission and more.')

-    parser.add_argument('-v', '--verbose', action='store_true', help='Increase console logging verbosity')
-    parser.add_argument('-q', '--quiet', action='store_true', help='Turn off console logging')
-    parser.add_argument('-d', '--daemon', action='store_true', help='Run as a daemon')
-    parser.add_argument('-p', '--port', type=int, help='Force Headphones to run on a specified port')
-    parser.add_argument('--datadir', help='Specify a directory where to store your data files')
+    parser.add_argument(
+        '-v', '--verbose', action='store_true', help='Increase console logging verbosity')
+    parser.add_argument(
+        '-q', '--quiet', action='store_true', help='Turn off console logging')
+    parser.add_argument(
+        '-d', '--daemon', action='store_true', help='Run as a daemon')
+    parser.add_argument(
+        '-p', '--port', type=int, help='Force Headphones to run on a specified port')
+    parser.add_argument(
+        '--datadir', help='Specify a directory where to store your data files')
    parser.add_argument('--config', help='Specify a config file to use')
-    parser.add_argument('--nolaunch', action='store_true', help='Prevent browser from launching on startup')
-    parser.add_argument('--pidfile', help='Create a pid file (only relevant when running as a daemon)')
+    parser.add_argument('--nolaunch', action='store_true',
+                        help='Prevent browser from launching on startup')
+    parser.add_argument(
+        '--pidfile', help='Create a pid file (only relevant when running as a daemon)')

    args = parser.parse_args()

@@ -81,9 +89,14 @@ def main():
    if args.quiet:
        headphones.QUIET = True

+    # Do an intial setup of the logger.
+    logger.initLogger(console=not headphones.QUIET, log_dir=False,
+        verbose=headphones.VERBOSE)
+
    if args.daemon:
        if sys.platform == 'win32':
-            sys.stderr.write("Daemonizing not supported under Windows, starting normally\n")
+            sys.stderr.write(
+                "Daemonizing not supported under Windows, starting normally\n")
        else:
            headphones.DAEMON = True
            headphones.QUIET = True
@@ -91,11 +104,14 @@ def main():
    if args.pidfile:
        headphones.PIDFILE = str(args.pidfile)

-        # If the pidfile already exists, headphones may still be running, so exit
+        # If the pidfile already exists, headphones may still be running, so
+        # exit
        if os.path.exists(headphones.PIDFILE):
-            sys.exit("PID file '" + headphones.PIDFILE + "' already exists. Exiting.")
+            raise SystemExit("PID file '%s' already exists. Exiting." %
+                headphones.PIDFILE)

-        # The pidfile is only useful in daemon mode, make sure we can write the file properly
+        # The pidfile is only useful in daemon mode, make sure we can write the
+        # file properly
        if headphones.DAEMON:
            headphones.CREATEPID = True

@@ -103,9 +119,10 @@ def main():
                with open(headphones.PIDFILE, 'w') as fp:
                    fp.write("pid\n")
            except IOError as e:
-                raise SystemExit("Unable to write PID file: %s [%d]", e.strerror, e.errno)
+                raise SystemExit("Unable to write PID file: %s", e)
        else:
-            logger.warn("Not running in daemon mode. PID file creation disabled.")
+            logger.warn("Not running in daemon mode. PID file creation " \
+                "disabled.")

    # Determine which data directory and config file to use
    if args.datadir:
@@ -114,27 +131,28 @@ def main():
        headphones.DATA_DIR = headphones.PROG_DIR

    if args.config:
-        headphones.CONFIG_FILE = args.config
+        config_file = args.config
    else:
-        headphones.CONFIG_FILE = os.path.join(headphones.DATA_DIR, 'config.ini')
+        config_file = os.path.join(headphones.DATA_DIR, 'config.ini')

    # Try to create the DATA_DIR if it doesn't exist
    if not os.path.exists(headphones.DATA_DIR):
        try:
            os.makedirs(headphones.DATA_DIR)
        except OSError:
-            raise SystemExit('Could not create data directory: ' + headphones.DATA_DIR + '. Exiting....')
+            raise SystemExit(
+                'Could not create data directory: ' + headphones.DATA_DIR + '. Exiting....')

    # Make sure the DATA_DIR is writeable
    if not os.access(headphones.DATA_DIR, os.W_OK):
-        raise SystemExit('Cannot write to the data directory: ' + headphones.DATA_DIR + '. Exiting...')
+        raise SystemExit(
+            'Cannot write to the data directory: ' + headphones.DATA_DIR + '. Exiting...')

    # Put the database in the DATA_DIR
    headphones.DB_FILE = os.path.join(headphones.DATA_DIR, 'headphones.db')
-    headphones.CFG = ConfigObj(headphones.CONFIG_FILE, encoding='utf-8')

    # Read config and start logging
-    headphones.initialize()
+    headphones.initialize(config_file)

    if headphones.DAEMON:
        headphones.daemonize()
@@ -147,28 +165,40 @@ def main():
        http_port = args.port
        logger.info('Using forced web server port: %i', http_port)
    else:
-        http_port = int(headphones.HTTP_PORT)
+        http_port = int(headphones.CONFIG.HTTP_PORT)
+
+    # Check if pyOpenSSL is installed. It is required for certificate generation
+    # and for CherryPy.
+    if headphones.CONFIG.ENABLE_HTTPS:
+        try:
+            import OpenSSL
+        except ImportError:
+            logger.warn("The pyOpenSSL module is missing. Install this " \
+                "module to enable HTTPS. HTTPS will be disabled.")
+            headphones.CONFIG.ENABLE_HTTPS = False

    # Try to start the server. Will exit here is address is already in use.
-    webstart.initialize({
+    web_config = {
        'http_port': http_port,
-        'http_host': headphones.HTTP_HOST,
-        'http_root': headphones.HTTP_ROOT,
-        'http_proxy': headphones.HTTP_PROXY,
-        'enable_https': headphones.ENABLE_HTTPS,
-        'https_cert': headphones.HTTPS_CERT,
-        'https_key': headphones.HTTPS_KEY,
-        'http_username': headphones.HTTP_USERNAME,
-        'http_password': headphones.HTTP_PASSWORD,
-    })
-
-    if headphones.LAUNCH_BROWSER and not args.nolaunch:
-        headphones.launch_browser(headphones.HTTP_HOST, http_port,
-            headphones.HTTP_ROOT)
+        'http_host': headphones.CONFIG.HTTP_HOST,
+        'http_root': headphones.CONFIG.HTTP_ROOT,
+        'http_proxy': headphones.CONFIG.HTTP_PROXY,
+        'enable_https': headphones.CONFIG.ENABLE_HTTPS,
+        'https_cert': headphones.CONFIG.HTTPS_CERT,
+        'https_key': headphones.CONFIG.HTTPS_KEY,
+        'http_username': headphones.CONFIG.HTTP_USERNAME,
+        'http_password': headphones.CONFIG.HTTP_PASSWORD,
+    }
+    webstart.initialize(web_config)

    # Start the background threads
    headphones.start()

+    # Open webbrowser
+    if headphones.CONFIG.LAUNCH_BROWSER and not args.nolaunch:
+        headphones.launch_browser(headphones.CONFIG.HTTP_HOST, http_port,
+                                  headphones.CONFIG.HTTP_ROOT)
+
    # Wait endlessy for a signal to happen
    while True:
        if not headphones.SIGNAL:
@@ -190,4 +220,4 @@ def main():

 # Call main()
 if __name__ == "__main__":
-    main()
+    main()
@@ -1,17 +1,18 @@
-#![preview thumb](https://github.com/rembo10/headphones/raw/master/data/images/headphoneslogo.png)Headphones
+#![Headphones Logo](https://github.com/rembo10/headphones/raw/master/data/images/headphoneslogo.png) Headphones

-###Support & Discuss
+Headphones is an automated music downloader for NZB and Torrent, written in Python. It supports SABnzbd, NZBget, Transmission, µTorrent and Blackhole.

-You are free to join the HP support community on IRC where you can ask questions, hang around and discuss anything related to HP.
+## Support & Discuss
+You are free to join the Headphones support community on IRC where you can ask questions, hang around and discuss anything related to HP.

-1. Use any IRC client and connect to the Freenode server.
-2. Join #headphones
+1. Use any IRC client and connect to the Freenode server, `irc.freenode.net`.
+2. Join the `#headphones` channel.

-###Installation and Notes
+## Installation and Notes

-[Read our Wiki](../../wiki) on how to install and use HeadPhones properly.
-
-[**Troubleshooting** page](../../wiki/TroubleShooting) in the wiki can help you with comon problems.
+* [Installation page](../../wiki/Installation) shows you how to install Headphones.
+* [Usage guide](../../wiki/Usage-guide) introduces you to Headphones.
+* [Troubleshooting page](../../wiki/TroubleShooting) in the wiki can help you with common problems.

 **Issues** can be reported on the GitHub issue tracker considering these rules:

@@ -27,17 +28,15 @@ You are free to join the HP support community on IRC where you can ask questions

 If you **comply with these rules** you can [post your request/issue](http://github.com/rembo10/headphones/issues).

-**Support** the project by implementing new features, solving support tickets and provide bug fixes. 
-If you change something in the code always make a PR to the developer branch instead of the master branch.
+**Support** the project by implementing new features, solving support tickets and provide bug fixes.

+## Screenshots

-###Screenshots
-
-Homepage (Artist Overview)
+Homepage (Artist Overview):

 ![preview thumb](http://i.imgur.com/LZO9a.png)

-One of the many settings pages....
+One of the many settings pages:

 ![preview thumb](http://i.imgur.com/xcWNy.png)

@@ -49,7 +48,7 @@ Import Your Favorite Artists:

 ![preview thumb](http://i.imgur.com/6tZoC.png)

-Artist Search Results (also search by album!):
+Search Results:

 ![preview thumb](http://i.imgur.com/rIV0P.png)

@@ -61,5 +60,5 @@ Album Page with track overview:

 ![preview thumb](http://i.imgur.com/kcjES.png)

-
-This is free software under the GPL v3 open source license - so feel free to do with it what you wish.
+## License
+This is free software under the GPL v3 open source license. Feel free to do with it what you wish, but any modification must be open sourced. A copy of the license is included.
@@ -0,0 +1,9 @@
+#!/bin/bash
+
+# Display information
+echo "This script will remove *.pyc files. These files are generated by Python, but they can cause conflicts after an upgrade. It's safe to remove them, because they will be regenerated."
+echo "Press enter to continue, or CTRL + C to quit."
+read
+
+# Remove the *.pyc
+find "`dirname $0`/.." -type f -name "*.pyc" -exec rm -rf {} \;
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+# Parameter check
+if [ -z "$1" ]; then
+    echo "Syntax: $0 <data directory>"
+    exit 1
+fi
+
+# Version file check
+if [ ! -s "$1/version.lock" ]; then
+    echo "Missing the version.lock file in the data folder, or the file is empty. Did you start Headphones at least once?"
+    exit 1
+fi
+
+# Git installation check
+if [ ! -x "$(command -v git)" ]; then
+    echo "Git is required to downgrade."
+    exit 1
+fi
+
+# Display information
+HASH=$(cat $1/version.lock)
+
+echo "This script will try to downgrade Headphones to the last version that started, version $HASH. Make sure you have a backup of your config file and database, just in case!"
+echo "Press enter to continue, or CTRL + C to quit."
+read
+
+# Downgrade
+cd "`dirname $0`/.."
+git reset --hard "$HASH"
+
+echo "All done, Headphones should be downgraded to the last version that started."
@@ -48,44 +48,44 @@
 					</div>
 				</div>
 			<a class="menu_link_edit" id="edit_search_term" href="#"><i class="fa fa-pencil"></i> Edit Search Term</a>
-				<div id="dialog2" title="Enter your own search term for this album" style="display:none" class="configtable">
-					<form action="editSearchTerm" method="GET" id="editSearchTerm">
-						<input type="hidden" name="AlbumID" value="${album['AlbumID']}">
-						<div class="row">
-							<%
-								if not album['SearchTerm']:
-									search_term = ""
-								else:
-									search_term = album['SearchTerm']
-								
-							%>
-							<input type="text" value="${search_term}" name="SearchTerm" size="40" />
-						</div>
-						<input type="button" value="Save changes" onclick="doAjaxCall('editSearchTerm',$(this),'tabs',true);return false;" data-success="Search term updated"/>
-					</form>
-				</div>	
+			<div id="dialog2" title="Enter your own search term for this album" style="display:none" class="configtable">
+				<form action="editSearchTerm" method="GET" id="editSearchTerm">
+					<input type="hidden" name="AlbumID" value="${album['AlbumID']}">
+					<div class="row">
+						<%
+							if not album['SearchTerm']:
+								search_term = ""
+							else:
+								search_term = album['SearchTerm']
+						%>
+						<input type="text" value="${search_term}" name="SearchTerm" size="40" />
+					</div>
+					<input type="button" value="Save changes" onclick="doAjaxCall('editSearchTerm',$(this),'tabs',true);return false;" data-success="Search term updated"/>
+				</form>
+			</div>
 			<a class="menu_link_edit" id="choose_specific_download" href="#" onclick="getAvailableDownloads()"><i class="fa fa-search"></i> Choose Specific Download</a>
-				<div id="choose_specific_download_dialog" title="Choose a specific download for this album" style="display:none" class="configtable">
-						<table class="display" id="downloads_table">
-							<thead>
-								<tr>
-									<th id="title">Title</th>
-									<th id="size">Size</th>
-									<th id="provider">Provider</th>
-									<th id="kind">Kind</th>
-								</tr>
-							</thead>
-							<tbody id="downloads_table_body">
-							</tbody>
-						</table>
-				</div>	
+			<div id="choose_specific_download_dialog" title="Choose a specific download for this album" style="display:none" class="configtable">
+				<table class="display" id="downloads_table">
+					<thead>
+						<tr>
+							<th id="title">Title</th>
+							<th id="size">Size</th>
+							<th id="provider">Provider</th>
+							<th id="kind">Kind</th>
+							<th id="matches" title="Matches quality and/or seed settings.">Matches</th>
+						</tr>
+					</thead>
+					<tbody id="downloads_table_body">
+					</tbody>
+				</table>
+			</div>
 		</div>
-	</div>	
+	</div>
 </%def>

 <%def name="body()">
 	<div class="table_wrapper">
-		<div id="albumheader" class="clearfix">	
+		<div id="albumheader" class="clearfix">
 			<div id="albumImg">
 				<img height="200" alt="" class="albumArt" src="artwork/album/${album['AlbumID']}">
 			</div>
@@ -197,14 +197,20 @@
 		var elem = $("#albumInfo");
 		getInfo(elem,id,'album');
 	}
-	
+
 	function initThisPage() {
 		$('#album_chooser').click(function() {
-			$('#dialog').dialog({ width: "500px" });
+			$('#dialog').dialog({
+				width: 500,
+				maxHeight: 500
+			});
 			return false;
 		});
 		$('#edit_search_term').click(function() {
-			$('#dialog2').dialog({ width: "500px" });
+			$('#dialog2').dialog({
+				width: 500,
+				maxHeight: 500
+			});
 			return false;
 		});
 		$('#refresh_artist').click(function() {
@@ -213,8 +219,8 @@
 		initActions();
 		setTimeout(function(){
 			initFancybox();
-			},1000);
-		
+		}, 1000);
+
 		$('#track_table').dataTable({
 			"aaSorting": [],
 			"bFilter": false,
@@ -231,21 +237,26 @@
 			feedback.fadeOut();
 			search_results = data
 			for( var i = 0, len = data.length; i < len; i++ ) {
-    			$('#downloads_table_body').append('<tr><td id="title"><a href="#" onclick="downloadSpecificRelease('+i+')">'+data[i].title+'</a></td><td id="size"><span title='+data[i].size+'></span>'+(data[i].size / (1024*1024)).toFixed(2)+' MB</td><td id="provider">'+data[i].provider+'</td><td id="kind">'+data[i].kind+'</td></tr>');
+    			$('#downloads_table_body').append('<tr><td id="title"><a href="#" onclick="downloadSpecificRelease('+i+')">'+data[i].title+'</a></td><td id="size"><span title='+data[i].size+'></span>'+(data[i].size / (1024*1024)).toFixed(2)+' MB</td><td id="provider">'+data[i].provider+'</td><td id="kind">'+data[i].kind+'</td><td id="matches">'+data[i].matches+'</td></tr>');
    		}
 			$('#downloads_table').dataTable({
 				"aoColumns": [
 					null,
-					{ "sType": "title-numeric"},
+					{"sType": "title-numeric"},
 					null,
-					null
+					null,
+					{"sType": "string"}
 				],
-				"aaSorting": [],
+				"aaSorting": [[ 4, 'desc']],
 				"bFilter": false,
 				"bInfo": false,
 				"bPaginate": false,
+				"bDestroy": true
 			});
-    		$("#choose_specific_download_dialog").dialog({ width: "1000px" });
+    		$("#choose_specific_download_dialog").dialog({
+    			width: "80%",
+    			maxHeight: 500
+    		});
    		return false;
 		});
 	}
@@ -326,6 +337,21 @@
        });
    }

+    jQuery.extend( jQuery.fn.dataTableExt.oSort, {
+		"title-numeric-pre": function ( a ) {
+			var x = a.match(/title="*(-?[0-9\.]+)/)[1];
+			return parseFloat( x );
+		},
+
+		"title-numeric-asc": function ( a, b ) {
+			return ((a < b) ? -1 : ((a > b) ? 1 : 0));
+		},
+
+        "title-numeric-desc": function ( a, b ) {
+            return ((a < b) ? 1 : ((a > b) ? -1 : 0));
+        }
+    } );
+
 	$(document).ready(function()	{
 		getAlbumInfo();
 		initThisPage();
@@ -10,6 +10,7 @@
 		<div id="subhead_menu">
 			<a id="menu_link_refresh" onclick="doSimpleAjaxCall('refreshArtist?ArtistID=${artist['ArtistID']}')" href="#"><i class="fa fa-refresh"></i> Refresh Artist</a>
 			<a id="menu_link_delete" href="deleteArtist?ArtistID=${artist['ArtistID']}"><i class="fa fa-trash-o"></i> Delete Artist</a>
+			<a id="menu_link_scan" onclick="doAjaxCall('scanArtist?ArtistID=${artist['ArtistID']}', $(this)),'table'" href="#" data-success="'${artist['ArtistName']}' was scanned"><i class="fa fa-refresh"></i> Scan Artist</a>
 			%if artist['Status'] == 'Paused':
 				<a id="menu_link_resume" href="#" onclick="doAjaxCall('resumeArtist?ArtistID=${artist['ArtistID']}',$(this),true)" data-success="${artist['ArtistName']} resumed"><i class="fa fa-play"></i> Resume Artist</a>
 			%else:
@@ -69,6 +70,7 @@
 				<th id="albumname">Name</th>
 				<th id="reldate">Date</th>
 				<th id="type">Type</th>
+				<th id="score">Metacritic</th>
 				<th id="status">Status</th>
 				<th id="have">Have</th>
 				<th id="bitrate">Bitrate</th>
@@ -124,6 +126,7 @@
 				<td id="albumname"><a href="albumPage?AlbumID=${album['AlbumID']}">${album['AlbumTitle']}</a></td>
 				<td id="reldate">${album['ReleaseDate']}</td>
 				<td id="type">${album['Type']}</td>
+				<td id="score">${album['CriticScore']}/${album['UserScore']}</td>
 				<td id="status">${album['Status']}
 				%if album['Status'] == 'Skipped' or album['Status'] == 'Ignored':
 					[<a href="#" onclick="doAjaxCall('queueAlbum?AlbumID=${album['AlbumID']}&ArtistID=${album['ArtistID']}',$(this),'table')" data-success="'${album['AlbumTitle']}' added to Wanted list">want</a>]
@@ -163,21 +166,21 @@
 	}

 	<%
-    if headphones.SONGKICK_FILTER_ENABLED:
+    if headphones.CONFIG.SONGKICK_FILTER_ENABLED:
        songkick_filter_enabled = "true"
    else:
        songkick_filter_enabled = "false"

-    if not headphones.SONGKICK_LOCATION:
+    if not headphones.CONFIG.SONGKICK_LOCATION:
    	songkick_location = "none"
    else:
-        songkick_location = headphones.SONGKICK_LOCATION
-    	
-    if headphones.SONGKICK_ENABLED:
+        songkick_location = headphones.CONFIG.SONGKICK_LOCATION
+
+    if headphones.CONFIG.SONGKICK_ENABLED:
        songkick_enabled = "true"
    else:
        songkick_enabled = "false"
-	
+
 	%>
 	function getArtistsCalendar() {
 		var template, calendarDomNode;
@@ -186,48 +189,48 @@

 		template = '<li><a target="_blank" href="URI"><span class="sk-name">NAME</span><span class="sk-location">LOC</span></a></li>';

-		$.getJSON("https://api.songkick.com/api/3.0/artists/mbid:${artist['ArtistID']}/calendar.json?apikey=${headphones.SONGKICK_APIKEY}&jsoncallback=?",
+		$.getJSON("https://api.songkick.com/api/3.0/artists/mbid:${artist['ArtistID']}/calendar.json?apikey=${headphones.CONFIG.SONGKICK_APIKEY}&jsoncallback=?",
 			function(data){
-				if (data['resultsPage'].totalEntries >= 1) {          
-          
-                    if( ${songkick_filter_enabled} ) {
+				if (data['resultsPage'].totalEntries >= 1) {
+                    if (${songkick_filter_enabled}) {
                        data.resultsPage.results.event = $.grep(data.resultsPage.results.event, function(element,index){
-                        return element.venue.metroArea.id == ${songkick_location};
-                    });
+                    	    return element.venue.metroArea.id == ${songkick_location};
+                    	});
                    }
-          
-					var tourDate;

-					calendarDomNode.show();
-					$("#artistImg").addClass('on-tour');
+                    if (data.resultsPage.results.event.length > 0) {
+						var tourDate;

-					jQuery.each( data['resultsPage'].results.event, function( i, event ) {
-						tourDate = template;
-						tourDate = tourDate.replace('URI',event.uri);
-						tourDate = tourDate.replace('NAME',event.displayName);
-						tourDate = tourDate.replace('LOC',event.location.city);
+						calendarDomNode.show();
+						$("#artistImg").addClass('on-tour');

-						calendarDomNode.append(tourDate);
-					});
+						jQuery.each(data.resultsPage.results.event, function(i, event) {
+							tourDate = template;
+							tourDate = tourDate.replace('URI',event.uri);
+							tourDate = tourDate.replace('NAME',event.displayName);
+							tourDate = tourDate.replace('LOC',event.location.city);

-					calendarDomNode.append('<li><img src="interfaces/default/images/songkick.png" alt="concerts by songkick" class="sk-logo" /></li>');
-					
-				$(function() {
-				  $("#artistCalendar").each(function() {
-					$("li:gt(4)", this).hide(); /* :gt() is zero-indexed */
-					$("li:nth-child(5)", this).after("<br><li class='more'><a href='#'>More...</a></li>"); /* :nth-child() is one-indexed */
-				  });
-				  $("li.more a").live("click", function() {
-					var li = $(this).parents("li:first");
-					li.parent().children().show();
-					li.remove();
-					return false;
-				  });
-				});
+							calendarDomNode.append(tourDate);
+						});

+						calendarDomNode.append('<li><img src="interfaces/default/images/songkick.png" alt="concerts by songkick" class="sk-logo" /></li>');
+
+						$(function() {
+						  $("#artistCalendar").each(function() {
+							$("li:gt(4)", this).hide(); /* :gt() is zero-indexed */
+							$("li:nth-child(5)", this).after("<br><li class='more'><a href='#'>More...</a></li>"); /* :nth-child() is one-indexed */
+						  });
+						  $("li.more").on("click", 'a', function() {
+							var li = $(this).parents("li:first");
+							li.parent().children().show();
+							li.remove();
+							return false;
+						  });
+						});
+					}
 				}
-			// data is JSON response object
-		});
+			}
+		);
 	}

 	var loadingMessage = false;
@@ -269,7 +272,7 @@
 			$('#dialog').dialog();
 			event.preventDefault();
 		});
-		$('#menu_link_modifyextra').click(function() {
+		$('#menu_link_modifyextra').click(function(event) {
 			$('#dialog').dialog();
 			event.preventDefault();
 		});
@@ -282,6 +285,7 @@
 				{ "sType": "date" },
 				null,
 				null,
+				null,
 				{ "sType": "title-numeric"},
 				null,
 				null
@@ -31,14 +31,14 @@
 <body>
 	<div id="container">
 		<div id="ajaxMsg" class="ajaxMsg"></div>
-		% if not headphones.CURRENT_VERSION:
+		% if headphones.CONFIG.CHECK_GITHUB and not headphones.CURRENT_VERSION:
 			<div id="updatebar">
 				You're running an unknown version of Headphones. <a href="update">Update</a> or
 				<a href="#" onclick="$('#updatebar').slideUp('slow');">Close</a>
 			</div>
-		% elif headphones.CURRENT_VERSION != headphones.LATEST_VERSION and headphones.COMMITS_BEHIND > 0 and headphones.INSTALL_TYPE != 'win':
+		% elif headphones.CONFIG.CHECK_GITHUB and headphones.CURRENT_VERSION != headphones.LATEST_VERSION and headphones.COMMITS_BEHIND > 0 and headphones.INSTALL_TYPE != 'win':
 			<div id="updatebar">
-				A <a href="https://github.com/${headphones.GIT_USER}/headphones/compare/${headphones.CURRENT_VERSION}...${headphones.LATEST_VERSION}"> newer version</a> is available. You're ${headphones.COMMITS_BEHIND} commits behind. <a href="update">Update</a> or <a href="#" onclick="$('#updatebar').slideUp('slow');">Close</a>
+				A <a href="https://github.com/${headphones.CONFIG.GIT_USER}/headphones/compare/${headphones.CURRENT_VERSION}...${headphones.LATEST_VERSION}"> newer version</a> is available. You're ${headphones.COMMITS_BEHIND} commits behind. <a href="update">Update</a> or <a href="#" onclick="$('#updatebar').slideUp('slow');">Close</a>
 			</div>
 		% endif

@@ -52,7 +52,7 @@
 				<li><a href="extras">extras</a></li>
 				<li><a href="manage">manage</a></li>
 				<li><a href="history">history</a></li>
-				<li><a href="logs" class="log">logs</a></li>
+				<li><a href="logs">logs</a></li>
 				<li><a href="config" class="config"><i class="fa fa-gear fa-lg"></i></a></li>
 			</ul>
 			<div id="searchbar">
@@ -62,6 +62,7 @@
 					<select name="type" id="search_type">
 						<option value="artist">Artist</option>
 						<option value="album">Album</option>
+						<option value="series">Series</option>
 					</select>
 					<input type="submit" value="Search"/>
 				</form>
@@ -96,8 +97,8 @@
 				%if version.HEADPHONES_VERSION != 'master':
 				(${version.HEADPHONES_VERSION})
 				%endif
-				%if headphones.GIT_BRANCH != 'master':
-				(${headphones.GIT_BRANCH})
+				%if headphones.CONFIG.GIT_BRANCH != 'master':
+				(${headphones.CONFIG.GIT_BRANCH})
 				%endif
 			</div>
 		</footer>
@@ -119,7 +120,7 @@
 	    $(document).ready(function() {
 	        $('form:first *:input[type!=hidden]:first').focus();
 	        try{
-	            var type = window.localStorage.getItem('search_type');
+	            var type = window.localStorage.getItem('search_type') || "artist";
 	            $("#search_type").val(type);
 	        } catch(e) {
 	        }
@@ -139,4 +140,4 @@

 <%def name="javascriptIncludes()"></%def>
 <%def name="headIncludes()"></%def>
-<%def name="headerIncludes()"></%def>
+<%def name="headerIncludes()"></%def>
@@ -50,13 +50,13 @@
                                    <label title="Username for web server authentication. Leave empty to disable.">
                                        HTTP Username
                                    </label>
-                                    <input type="text" name="http_username" value="${config['http_user']}" size="30">
+                                    <input type="text" name="http_username" value="${config['http_username']}" size="30">
                                </div>
                                <div class="row">
                                    <label title="Password for web server authentication. Leave empty to disable.">
                                        HTTP Password
                                    </label>
-                                    <input type="password" name="http_password" value="${config['http_pass']}" size="30">
+                                    <input type="password" name="http_password" value="${config['http_password']}" size="30">
                                </div>
                                <div class="row checkbox">
                                    <input type="checkbox" name="launch_browser" value="1" ${config['launch_browser']} />
@@ -86,7 +86,7 @@
                            <fieldset>
                                <legend>API</legend>
                                <div class="row checkbox">
-                                    <input type="checkbox" id="useapi" name="api_enabled" id="api_enabled" value="1" ${config['api_enabled']} />
+                                    <input type="checkbox" id="api_enabled" name="api_enabled" value="1" ${config['api_enabled']} />
                                    <label title="Allow remote applications to interface with Headphones">
                                        Enable API
                                    </label>
@@ -101,6 +101,7 @@

                            <fieldset>
                                <legend>Interval</legend>
+                                <small>An interval of 0 will disable a task.</small>
                                <div class="row">
                                    <label title="Time between two searches for new downloads.">
                                        Search Interval
@@ -137,7 +138,7 @@
                    </tr>
                    <tr>
                        <div class="configmessage">
-                            <i class="fa fa-info-circle"></i> Web Interface changes require a restart to take effect
+                            <i class="fa fa-info-circle"></i> Web Interface changes require a restart to take effect. Saving settings will restart intervals if changed.
                        </div>
                    </tr>
                </table>
@@ -166,25 +167,25 @@
                                    <label title="SABnzbd username. Leave empty if not applicable.">
                                        SABnzbd Username
                                    </label>
-                                    <input type="text" name="sab_username" value="${config['sab_user']}" size="20">
+                                    <input type="text" name="sab_username" value="${config['sab_username']}" size="20">
                                </div>
                                <div class="row">
                                    <label title="SABnzbd password. Leave empty if not applicable.">
                                        SABnzbd Password
                                    </label>
-                                    <input type="password" name="sab_password" value="${config['sab_pass']}" size="20">
+                                    <input type="password" name="sab_password" value="${config['sab_password']}" size="20">
                                </div>
                                <div class="row">
                                    <label title="SABnzbd API key. Can be found in SABnzbd settings.">
                                        SABnzbd API key
                                    </label>
-                                    <input type="text" name="sab_apikey" value="${config['sab_api']}" size="36">
+                                    <input type="text" name="sab_apikey" value="${config['sab_apikey']}" size="36">
                                </div>
                                <div class="row">
                                    <label title="Name of SABnzbd category to add downloads to.">
                                        SABnzbd Category
                                    </label>
-                                    <input type="text" name="sab_category" value="${config['sab_cat']}" size="20">
+                                    <input type="text" name="sab_category" value="${config['sab_category']}" size="20">
                                </div>
                            </fieldset>

@@ -200,19 +201,19 @@
                                    <label title="NZBGet username. Leave empty if not applicable">
                                        NZBget Username
                                    </label>
-                                    <input type="text" name="nzbget_username" value="${config['nzbget_user']}" size="20">
+                                    <input type="text" name="nzbget_username" value="${config['nzbget_username']}" size="20">
                                </div>
                                <div class="row">
                                    <label title="NZBGet password. Leave empty if not applicable">
                                        NZBget Password
                                    </label>
-                                    <input type="password" name="nzbget_password" value="${config['nzbget_pass']}" size="20">
+                                    <input type="password" name="nzbget_password" value="${config['nzbget_password']}" size="20">
                                </div>
                                <div class="row">
                                    <label title="Name of NZBget category to add downloads to.">
                                        NZBget Category
                                    </label>
-                                    <input type="text" name="nzbget_category" value="${config['nzbget_cat']}" size="20">
+                                    <input type="text" name="nzbget_category" value="${config['nzbget_category']}" size="20">
                                </div>
                                <%
                                    if config['nzbget_priority'] == -100:
@@ -266,7 +267,7 @@
                                        prio_force = ''
                                %>
                                <div class="row">
-                                    <label>NZBget Priority:</label>
+                                    <label>NZBget Priority</label>
                                    <select name="nzbget_priority" id="nzbget_priority">
                                        <option value="-100" ${prio_verylow}>Very Low</option>
                                        <option value="-50" ${prio_low}>Low</option>
@@ -294,7 +295,7 @@
                                        Music Download Directory:
                                    </label>
                                    <input type="text" name="download_dir" value="${config['download_dir']}" size="50">
-                                    <small>Full path where SAB or NZBget downloads your music. e.g. /Users/name/Downloads/music</small>
+                                    <small>Full path where SAB or NZBget downloads your music, e.g. /Users/name/Downloads/music</small>
                                </div>
                                <div class="row">
                                    <label title="Number of days of retention your usenet provider provides.">
@@ -317,10 +318,27 @@
                                    <input type="text" name="torrentblackhole_dir" value="${config['torrentblackhole_dir']}" size="50">
                                    <small>Folder your Download program watches for Torrents</small>
                                </div>
-                                <div class="row checkbox">
-                                    <label>Open Magnet Links</label>
-                                    <input type="checkbox" name="open_magnet_links" value="1" ${config['open_magnet_links']}>
-                                    <small>Allow Headphones to open magnet links</small>
+
+                                <div class="row">
+                                    <label>Magnet links</label>
+
+                                    <label class="inline" title="Invoke shell command to open magnet URL.">
+                                        <input type="radio" name="magnet_links" id="magnet_links_0" value="0" ${config['magnet_links_0']}>
+                                        Ignore
+                                    </label>
+
+                                    <label class="inline" title="Use external service to convert magnet links into torrents.">
+                                        <input type="radio" name="magnet_links" id="magnet_links_1" value="1" ${config['magnet_links_1']}>
+                                        Open
+                                    </label>
+
+                                    <label class="inline">
+                                        <input type="radio" name="magnet_links" id="magnet_links_2" value="2" ${config['magnet_links_2']}>
+                                        Convert
+                                    </label>
+                                    <div style="clear: both"></div>
+
+                                    <small>Note: opening magnet URL's is not suitable for headless/console/terminal servers.</small>
                                </div>
                            </fieldset>
                            <fieldset id="transmission_options">
@@ -331,11 +349,11 @@
                                </div>
                                <div class="row">
                                    <label>Transmission Username</label>
-                                    <input type="text" name="transmission_username" value="${config['transmission_user']}" size="30">
+                                    <input type="text" name="transmission_username" value="${config['transmission_username']}" size="30">
                                </div>
                                <div class="row">
                                    <label>Transmission Password</label>
-                                    <input type="password" name="transmission_password" value="${config['transmission_pass']}" size="30">
+                                    <input type="password" name="transmission_password" value="${config['transmission_password']}" size="30">
                                </div>
                                <div class="row">
                                    <small>Note: With Transmission, you can specify a different download directory for downloads sent from Headphones.
@@ -351,11 +369,11 @@
                                </div>
                                <div class="row">
                                    <label>uTorrent Username</label>
-                                    <input type="text" name="utorrent_username" value="${config['utorrent_user']}" size="30">
+                                    <input type="text" name="utorrent_username" value="${config['utorrent_username']}" size="30">
                                </div>
                                <div class="row">
                                    <label>uTorrent Password</label>
-                                    <input type="password" name="utorrent_password" value="${config['utorrent_pass']}" size="30">
+                                    <input type="password" name="utorrent_password" value="${config['utorrent_password']}" size="30">
                                </div>
                                <div class="row">
                                    <label>uTorrent Label</label>
@@ -364,14 +382,14 @@
                            </fieldset>
                            <fieldset id="general_torrent_options">
                                <div class="row">
-                                    <label>Minimum seeders:</label>
+                                    <label>Minimum seeders</label>
                                    <input type="text" name="numberofseeders" value="${config['numberofseeders']}" size="5">
                                    <small>Number of minimum seeders a torrent must have to be accepted</small>
                                </div>
                                <div class="row">
                                    <label>Music Download Directory</label>
                                    <input type="text" name="download_torrent_dir" value="${config['download_torrent_dir']}" size="50">
-                                    <small>Full path where your torrent client downloads your music e.g. '/Users/name/Downloads/music'</small>
+                                    <small>Full path where your torrent client downloads your music e.g. /Users/name/Downloads/music.</small>
                                </div>
                                <div class="row checkbox">
                                    <label>Keep Files for Seeding</label>
@@ -382,7 +400,7 @@
                                    <input type="radio" name="prefer_torrents" id="prefer_torrents_0" value="0" ${config['prefer_torrents_0']}>NZBs
                                    <input type="radio" name="prefer_torrents" id="prefer_torrents_1" value="1" ${config['prefer_torrents_1']}>Torrents
                                    <input type="radio" name="prefer_torrents" id="prefer_torrents_2" value="2" ${config['prefer_torrents_2']}>No Preference
-                                    </div>
+                                </div>
                            </fieldset>
                        </td>
                    </tr>
@@ -397,7 +415,8 @@
                            <fieldset>
                                <legend>Headphones Indexer</legend>
                                <div class="row checkbox">
-                                    <input id="use_headphones_indexer" type="checkbox" name="use_headphones_indexer" onclick="initConfigCheckbox($(this));" value="1" ${config['use_headphones_indexer']} /><label>Use Headphones Indexer</label>
+                                    <input id="headphones_indexer" type="checkbox" name="headphones_indexer" onclick="initConfigCheckbox($(this));" value="1" ${config['headphones_indexer']} />
+                                    <label>Use Headphones Indexer</label>
                                </div>
                                <div class="config">
                                    <div class="row">
@@ -410,14 +429,14 @@
                                        <input class="hppass" type="password" value="${config['hppass']}" size="20">
                                    </div>
                                    <div class="row">
-                                        <a href="http://headphones.codeshy.com/vip" id="vipserver" target="_blank">Don't have an account? Sign up!</a>
+                                        <a href="https://headphones.codeshy.com/vip" id="vipserver" target="_blank">Don't have an account? Sign up!</a>
                                    </div>
                                </div>
                            </fieldset>
                            <fieldset>
                                <legend>Custom Newznab Providers</legend>
                                <div class="row checkbox">
-                                    <input  id="usenewznab" type="checkbox" name="newznab" onclick="initConfigCheckbox($(this));" value="1" ${config['use_newznab']} /><label>Use Newznab</label>
+                                    <input id="use_newznab" type="checkbox" name="use_newznab" onclick="initConfigCheckbox($(this));" value="1" ${config['use_newznab']} /><label>Use Newznab</label>
                                </div>
                                <div id="newznab_providers">
                                    <div class="config" id="newznab1">
@@ -428,7 +447,7 @@
                                        </div>
                                        <div class="row">
                                            <label>Newznab API</label>
-                                            <input type="text" name="newznab_apikey" value="${config['newznab_api']}" size="36">
+                                            <input type="text" name="newznab_apikey" value="${config['newznab_apikey']}" size="36">
                                        </div>
                                        <div class="row checkbox">
                                            <input id="newznab_enabled" type="checkbox" name="newznab_enabled" onclick="initConfigCheckbox($(this));" value="1" ${config['newznab_enabled']} /><label>Enabled</label>
@@ -471,7 +490,7 @@
                            <fieldset>
                                <legend>NZBs.org</legend>
                                <div class="row checkbox">
-                                    <input id="usenzbsorg" type="checkbox" name="nzbsorg" onclick="initConfigCheckbox($(this));" value="1" ${config['use_nzbsorg']} /><label>Use NZBs.org</label>
+                                    <input id="use_nzbsorg" type="checkbox" name="use_nzbsorg" onclick="initConfigCheckbox($(this));" value="1" ${config['use_nzbsorg']} /><label>Use NZBs.org</label>
                                </div>
                                <div class="config">
                                    <div class="row">
@@ -483,7 +502,7 @@
                            <fieldset>
                                <legend>omgwtfnzbs</legend>
                                <div class="row checkbox">
-                                    <input id="useomgwtfnzbs" type="checkbox" name="omgwtfnzbs" onclick="initConfigCheckbox($(this));" value="1" ${config['use_omgwtfnzbs']} /><label>Use omgwtfnzbs</label>
+                                    <input id="use_omgwtfnzbs" type="checkbox" name="use_omgwtfnzbs" onclick="initConfigCheckbox($(this));" value="1" ${config['use_omgwtfnzbs']} /><label>Use omgwtfnzbs</label>
                                </div>
                                <div class="config">
                                    <div class="row">
@@ -504,16 +523,34 @@
                                <fieldset>
                                    <legend>The Pirate Bay</legend>
                                    <div class="row checkbox">
-                                        <input id="usepiratebay" type="checkbox" name="use_piratebay" value="1" ${config['use_piratebay']} /><label>Use The Pirate Bay</label>
+                                        <input id="use_piratebay" type="checkbox" name="use_piratebay" value="1" ${config['use_piratebay']} /><label>Use The Pirate Bay</label>
                                    </div>
                                    <div class="config">
                                        <div class="row">
-                                            <label>Proxy URL (Optional): </label>
+                                            <label>Proxy URL</label>
                                            <input type="text" name="piratebay_proxy_url" value="${config['piratebay_proxy_url']}" size="36">
+                                            <small>Optional. Leave empty for default.</small>
                                        </div>
                                        <div class="row">
-                                            <label>Seed Ratio: </label>
-                                            <input type="text" class="override-float" name="piratebay_ratio" value="${config['piratebay_ratio']}" size="10" title="Stop seeding when ratio met, 0 = unlimited. Scheduled job will remove torrent when post processed and finished seeding">
+                                            <label>Seed Ratio</label>
+                                            <input type="text" class="override-float" name="piratebay_ratio" value="${config['piratebay_ratio']}" size="10" title="Stop seeding when ratio met, 0 = unlimited. Scheduled job will remove torrent when post processed and finished seeding.">
+                                        </div>
+                                    </div>
+                                </fieldset>
+
+                                <fieldset>
+                                    <legend>Old Pirate Bay</legend>
+                                    <div class="row checkbox">
+                                        <input id="use_oldpiratebay" type="checkbox" name="use_oldpiratebay" value="1" ${config['use_oldpiratebay']} /><label>Use Old Pirate Bay</label>
+                                    </div>
+                                    <div class="config">
+                                        <div class="row">
+                                            <label>URL</label>
+                                            <input type="text" name="oldpiratebay_url" value="${config['oldpiratebay_url']}" size="36">
+                                        </div>
+                                        <div class="row">
+                                            <label>Seed Ratio</label>
+                                            <input type="text" class="override-float" name="oldpiratebay_ratio" value="${config['oldpiratebay_ratio']}" size="10" title="Stop seeding when ratio met, 0 = unlimited. Scheduled job will remove torrent when post processed and finished seeding.">
                                        </div>
                                    </div>
                                </fieldset>
@@ -521,15 +558,16 @@
                                <fieldset>
                                    <legend>Kick Ass Torrents</legend>
                                    <div class="row checkbox">
-                                        <input id="usekat" type="checkbox" name="use_kat" value="1" ${config['use_kat']} /><label>Use Kick Ass Torrents</label>
+                                        <input id="use_kat" type="checkbox" name="use_kat" value="1" ${config['use_kat']} /><label>Use Kick Ass Torrents</label>
                                    </div>
                                    <div class="config">
                                        <div class="row">
-                                            <label>Proxy URL (Optional): </label>
+                                            <label>Proxy URL</label>
                                            <input type="text" name="kat_proxy_url" value="${config['kat_proxy_url']}" size="36">
+                                            <small>Optional. Leave empty for default.</small>
                                        </div>
                                        <div class="row">
-                                            <label>Seed Ratio: </label>
+                                            <label>Seed Ratio</label>
                                            <input type="text" class="override-float" name="kat_ratio" value="${config['kat_ratio']}" size="10" title="Stop seeding when ratio met, 0 = unlimited. Scheduled job will remove torrent when post processed and finished seeding">
                                        </div>
                                    </div>
@@ -538,19 +576,19 @@
                                <fieldset>
                                    <legend>Waffles.fm</legend>
                                    <div class="row checkbox">
-                                        <input id="usewaffles" type="checkbox" name="waffles" onclick="initConfigCheckbox($(this));" value="1" ${config['use_waffles']} /><label>Use Waffles.fm</label>
+                                        <input id="use_waffles" type="checkbox" name="use_waffles" onclick="initConfigCheckbox($(this));" value="1" ${config['use_waffles']} /><label>Use Waffles.fm</label>
                                    </div>
                                    <div class="config">
                                        <div class="row">
-                                            <label>UID Number: </label>
+                                            <label>UID Number</label>
                                            <input type="text" name="waffles_uid" value="${config['waffles_uid']}" size="36">
                                        </div>
                                        <div class="row">
-                                            <label>Passkey: </label>
+                                            <label>Passkey</label>
                                            <input type="password" name="waffles_passkey" value="${config['waffles_passkey']}" size="36">
                                        </div>
                                        <div class="row">
-                                            <label>Seed Ratio: </label>
+                                            <label>Seed Ratio</label>
                                            <input type="text" class="override-float" name="waffles_ratio" value="${config['waffles_ratio']}" size="10" title="Stop seeding when ratio met, 0 = unlimited. Scheduled job will remove torrent when post processed and finished seeding">
                                        </div>
                                    </div>
@@ -559,19 +597,19 @@
                                <fieldset>
                                    <legend>rutracker.org</legend>
                                    <div class="row checkbox">
-                                        <input id="userutracker" type="checkbox" name="rutracker" onclick="initConfigCheckbox($(this));" value="1" ${config['use_rutracker']} /><label>Use rutracker.org</label>
+                                        <input id="use_rutracker" type="checkbox" name="use_rutracker" onclick="initConfigCheckbox($(this));" value="1" ${config['use_rutracker']} /><label>Use rutracker.org</label>
                                    </div>
                                    <div class="config">
                                        <div class="row">
-                                            <label>User Name: </label>
+                                            <label>Userame</label>
                                            <input type="text" name="rutracker_user" value="${config['rutracker_user']}" size="36">
                                        </div>
                                        <div class="row">
-                                            <label>Password: </label>
+                                            <label>Password</label>
                                            <input type="password" name="rutracker_password" value="${config['rutracker_password']}" size="36">
                                        </div>
                                        <div class="row">
-                                            <label>Seed Ratio: </label>
+                                            <label>Seed Ratio</label>
                                            <input type="text" class="override-float" name="rutracker_ratio" value="${config['rutracker_ratio']}" size="10" title="Stop seeding when ratio met, 0 = unlimited. Scheduled job will remove torrent when post processed and finished seeding">
                                        </div>
                                    </div>
@@ -580,7 +618,7 @@
                                <fieldset>
                                    <legend>What.cd</legend>
                                    <div class="row checkbox">
-                                        <input id="usewhatcd" type="checkbox" name="whatcd" onclick="initConfigCheckbox($(this));" value="1" ${config['use_whatcd']} /><label>Use What.cd</label>
+                                        <input id="use_whatcd" type="checkbox" name="use_whatcd" onclick="initConfigCheckbox($(this));" value="1" ${config['use_whatcd']} /><label>Use What.cd</label>
                                    </div>
                                    <div class="config">
                                        <div class="row">
@@ -601,7 +639,7 @@
                                <fieldset>
                                    <legend>Mininova</legend>
                                     <div class="row checkbox">
-                                        <input id="usemininova" type="checkbox" name="use_mininova" value="1" ${config['use_mininova']} /><label>Use Mininova</label>
+                                        <input id="use_mininova" type="checkbox" name="use_mininova" value="1" ${config['use_mininova']} /><label>Use Mininova</label>
                                    </div>
                                    <div class="config">
                                        <div class="row">
@@ -650,15 +688,15 @@
                                    <div id="preferred_bitrate_options">
                                        <div class="row">
                                            Target bitrate
-                                            <input type="text" class="override-float" name="preferred_bitrate" value="${config['pref_bitrate']}" size="3">kbps<br>
+                                            <input type="text" class="override-float" name="preferred_bitrate" value="${config['preferred_bitrate']}" size="3">kbps<br>
                                        </div>
                                        <div class="row">
                                            <span style="padding-left: 20px">
-                                                Reject if <strong>less than</strong> <input type="text" class="override-float" name="preferred_bitrate_low_buffer" value="${config['pref_bitrate_low']}" size="3">% or <strong>more than</strong> <input type="text" class="override-float" name="preferred_bitrate_high_buffer" value="${config['pref_bitrate_high']}" size="3">% of the target size (leave blank for no limit)<br>
+                                                Reject if <strong>less than</strong> <input type="text" class="override-float" name="preferred_bitrate_low_buffer" value="${config['preferred_bitrate_low']}" size="3">% or <strong>more than</strong> <input type="text" class="override-float" name="preferred_bitrate_high_buffer" value="${config['preferred_bitrate_high']}" size="3">% of the target size (leave blank for no limit)<br>
                                            </span>
                                        </div>
                                        <div class="row checkbox left" style="padding-left: 20px">
-                                            <input type="checkbox" name="preferred_bitrate_allow_lossless" value="1" ${config['pref_bitrate_allow_lossless']}>
+                                            <input type="checkbox" name="preferred_bitrate_allow_lossless" value="1" ${config['preferred_bitrate_allow_lossless']}>
                                            <label>Allow lossless if no good lossy match found</label>
                                        </div>
                                        <div class="row checkbox left" style="padding-left: 20px">
@@ -670,7 +708,7 @@
                            </fieldset>
                            <fieldset>
                                <legend>Search Words</legend>
-                                <small>Separate words with a comma, e.g. "word1,word2,word3"</small>
+                                <small>Separate words with a comma, e.g. "word1,word2,word3".</small>
                                <div class="row">
                                    <label>Ignored Words</label>
                                    <input type="text" name="ignored_words" value="${config['ignored_words']}" size="50">
@@ -686,16 +724,24 @@
                                    <input type="text" name="required_words" value="${config['required_words']}" size="50">
                                    <small>Results without these words in the title will be filtered out. You can use OR: 'flac OR lossless OR alac, vinyl'</small>
                                </div>
+                                <div class="row checkbox left clearfix">
+                                    <label title="Filter out releases that contain the words 'clean','edited' or 'censored', as long as those words aren't in the search term">
+                                        Ignore clean/censored releases
+                                        <input type="checkbox" name="ignore_clean_releases" id="ignore_clean_releases" value="1" ${config['ignore_clean_releases']} />
+                                    </label>
+                                </div>
                            </fieldset>
                        </td>
                        <td>
                            <fieldset>
                                <legend>Post-Processing</legend>
                                <div class="row checkbox left clearfix">
-                                    <label title="Freeze the database, so new artists won't be added automatically. Use this if Headphones adds artists because due to wrong snatches. This check is skipped when the folder name is appended with release group ID.">
-                                        Freeze database for adding new artist
-                                        <input type="checkbox" name="freeze_db" id="freeze_db" value="1" ${config['freeze_db']} />
+                                    <label title="Use associated .cue sheet to split single file albums into multiple tracks. Requires shntool with flac or xld cli (OS X) to be installed.">
+                                        Split single file albums into multiple tracks
+                                        <input type="checkbox" name="cue_split" id="cue_split" value="1" ${config['cue_split']} />
                                    </label>
+                                </div>
+                                <div class="row checkbox left clearfix">
                                    <label>
                                        Move downloads to Destination Folder
                                        <input type="checkbox" name="move_files" id="move_files" value="1" ${config['move_files']} />
@@ -707,6 +753,12 @@
                                                <input type="checkbox" name="replace_existing_folders" value="1" ${config['replace_existing_folders']}>
                                            </label>
                                         </div>
+                                         <div class="row indent">
+                                            <label>
+                                                Keep original folder?
+                                                <input type="checkbox" name="keep_original_folder" value="1" ${config['keep_original_folder']}>
+                                            </label>
+                                         </div>
                                    </div>
                                    <label>
                                        Rename files
@@ -742,16 +794,24 @@
                                        Embed lyrics
                                        <input type="checkbox" name="embed_lyrics" value="1" ${config['embed_lyrics']}>
                                    </label>
+                                    <label title="Use ID3v2.3 instead of ID3v2.4">
+                                        Tag using ID3v2.3
+                                        <input type="checkbox" name="idtag" value="1" ${config['idtag']}>
+                                    </label>
                                </div>
                                <div class="row">
-                                    <label>Path to Destination Folder</label>
-                                    <input type="text" name="destination_dir" value="${config['dest_dir']}" size="50">
-                                    <small>e.g. /Users/name/Music/iTunes or /Volumes/share/music</small>
+                                    <label>
+                                        Destination Directory:
+                                    </label>
+                                    <input type="text" name="destination_dir" value="${config['destination_dir']}" size="50">
+                                    <small>The directory where Headphones will move file to after post processing, e.g. /Volumes/share/music.</small>
                                </div>
                                <div class="row">
-                                    <label>Path to Lossless Destination folder (optional)</label>
-                                    <input type="text" name="lossless_destination_dir" value="${config['lossless_dest_dir']}" size="50">
-                                    <small>Set this if you have a separate directory for lossless music</small>
+                                    <label>
+                                        Lossless Destination Directory:
+                                    </label>
+                                    <input type="text" name="lossless_destination_dir" value="${config['lossless_destination_dir']}" size="50">
+                                    <small>Optional. Set this if you have a separate directory for lossless music.</small>
                                </div>
                            </fieldset>
                        </td>
@@ -793,7 +853,7 @@
                                        <input type="checkbox" name="prowl_onsnatch" value="1" ${config['prowl_onsnatch']} /><label>Notify on snatch?</label>
                                    </div>
                                    <div class="row">
-                                        <label>Priority (-2,-1,0,1 or 2):</label>
+                                        <label>Priority (-2,-1,0,1 or 2)</label>
                                        <input type="text" name="prowl_priority" value="${config['prowl_priority']}" size="2">
                                    </div>
                                </div>
@@ -847,6 +907,10 @@
                                        <label>Plex Password</label><input type="password" name="plex_password" value="${config['plex_password']}" size="30">
 					                    <small>Password of your Plex client API (blank for none)</small>
                                    </div>
+                                    <div class="row">
+                                        <label>Plex Token</label><input type="text" name="plex_token" value="${config['plex_token']}" size="30">
+                              <small>Plex Token (for use with Plex Home)</small>
+                                    </div>
                                    <div class="checkbox row">
                                        <input type="checkbox" name="plex_update" value="1" ${config['plex_update']} /><label>Update Plex Library</label>
                                    </div>
@@ -968,11 +1032,11 @@
                                        <input type="checkbox" name="pushover_onsnatch" value="1" ${config['pushover_onsnatch']} /><label>Notify on snatch?</label>
                                    </div>
                                    <div class="row">
-                                        <label>Priority (-1,0, or 1):</label>
+                                        <label>Priority (-1,0, or 1)</label>
                                        <input type="text" name="pushover_priority" value="${config['pushover_priority']}" size="2">
                                    </div>
                                    <div class="row">
-                                        <label>API Token (leave blank to use Headphones default):</label><input type="text" name="pushover_apitoken" value="${config['pushover_apitoken']}" size="50">
+                                        <label>API Token (leave blank to use Headphones default)</label><input type="text" name="pushover_apitoken" value="${config['pushover_apitoken']}" size="50">
                                    </div>
                                </div>
                            </fieldset>
@@ -1056,6 +1120,42 @@
                                </div>
                            </fieldset>

+                           <fieldset>
+                                <h3>Email</h3>
+                                <div class="row checkbox">
+                                    <input type="checkbox" name="email_enabled" id="email" value="1" ${config['email_enabled']} /><label>Enable Email Notifications</label>
+                                </div>
+                                <div id="email_options">
+                                    <div class="row">
+                                        <label>From</label><input type="text" name="email_from" value="${config['email_from']}" size="254">
+                                    </div>
+                                    <div class="row">
+                                        <label>To</label><input type="text" name="email_to" value="${config['email_to']}" size="254">
+                                    </div>
+                                    <div class="row">
+                                        <label>SMTP Server</label><input type="text" name="email_smtp_server" value="${config['email_smtp_server']}" size="254">
+                                    </div>
+                                    <div class="row">
+                                        <label>SMTP User</label><input type="text" name="email_smtp_user" value="${config['email_smtp_user']}" size="254">
+                                    </div>
+                                    <div class="row">
+                                        <label>SMTP Password</label><input type="password" name="email_smtp_password" value="${config['email_smtp_password']}" size="50">
+                                    </div>
+                                    <div class="row checkbox">
+                                        <input type="text" class="override-float" name="email_smtp_port" value="${config['email_smtp_port']}" size="4"><label>SMTP Port</label>
+                                    </div>
+                                    <div class="row checkbox">
+                                        <input type="checkbox" name="email_ssl" value="1" ${config['email_ssl']} /><label>SSL</label>
+                                    </div>
+                                    <div class="row checkbox">
+                                        <input type="checkbox" name="email_tls" value="1" ${config['email_tls']} /><label>TLS</label>
+                                    </div>
+                                    <div class="row checkbox">
+                                        <input type="checkbox" name="email_onsnatch" value="1" ${config['email_onsnatch']} /><label>Notify on snatch?</label>
+                                    </div>
+                                </div>
+                            </fieldset>
+
                            <fieldset>
                            </td>
                    </tr>
@@ -1140,7 +1240,7 @@
                            </div>
                            <div class="row">
                                <div class="row">
-                                    <label>Multi-core count:</label>
+                                    <label>Multi-core count</label>
                                    <input type="text" name="encoder_multicore_count" value="${config['encoder_multicore_count']}" size="7">
                                    <small>Set equal to the number of cores, or 0 for auto</small>
                                </div>
@@ -1241,7 +1341,7 @@
                            </div>
                            <div class="row">
                                <label>Path to Encoder</label>
-                                <input type="text" name="encoderfolder" value="${config['encoderfolder']}" size="43">
+                                <input type="text" name="encoder_path" value="${config['encoder_path']}" size="43">
                            </div>
                        </td>
                        <td>
@@ -1278,16 +1378,27 @@
                                <div class="row left checkbox">
                                    <input type="checkbox" name="autowant_all" value="1" ${config['autowant_all']} /><label>Automatically mark all albums as wanted</label>
                                </div>
+                                <div class="row left checkbox">
+                                    <input type="checkbox" title="Automatically mark manually added albums from search results as wanted"
+                                           name="autowant_manually_added" value="1" ${config['autowant_manually_added']} />
+                                    <label>Automatically mark manually added albums as wanted</label>
+                                </div>
+                                <div class="row left checkbox">
+                                    <label title="Freeze the database, so new artists won't be added automatically. Use this if Headphones adds artists because due to wrong snatches. This check is skipped when the folder name is appended with release group ID.">
+                                        Don't add new artists when post-processing albums
+                                        <input type="checkbox" name="freeze_db" id="freeze_db" value="1" ${config['freeze_db']} />
+                                    </label>
+                                </div>
                                <div class="row">
-                                    <label>Folder Permissions:</label>
+                                    <label>Folder Permissions</label>
                                    <input type="text" name="folder_permissions" value="${config['folder_permissions']}" size="7">
                                </div>
                                <div class="row">
-                                    <label>File Permissions:</label>
+                                    <label>File Permissions</label>
                                    <input type="text" name="file_permissions" value="${config['file_permissions']}" size="7">
                                </div>
                                <div class="row">
-                                    <label>Cache Size (in MB):</label>
+                                    <label>Cache Size (in MB)</label>
                                    <input type="text" name="cache_sizemb" value="${config['cache_sizemb']}" size="7">
                                </div>
                            </fieldset>
@@ -1298,7 +1409,7 @@
                                    <select name="interface"><h3>
                                    %for interface in config['interface_list']:
                                    <%
-                                        if interface == headphones.INTERFACE:
+                                        if interface == headphones.CONFIG.INTERFACE:
                                            selected = 'selected="selected"'
                                        else:
                                            selected = ''
@@ -1308,11 +1419,11 @@
                                    </select>
                                </div>
                                <div class="row">
-                                    <label>Log Directory:</label>
+                                    <label>Log Directory</label>
                                    <input type="text" name="log_dir" value="${config['log_dir']}" size="50">
                                </div>
                                <div class="row">
-                                    <label>Cache Directory:</label>
+                                    <label>Cache Directory</label>
                                    <input type="text" name="cache_dir" value="${config['cache_dir']}" size="50">
                                </div>
                            </fieldset>
@@ -1324,7 +1435,7 @@
                                </div>
                                <div id="songkickoptions">
                                  <div class="row">
-                                      <label>API Key:</label>
+                                      <label>API Key</label>
                                      <input type="text" name="songkick_apikey" value="${config['songkick_apikey']}" size="50">
                                  </div>
                                  <div class="row checkbox">
@@ -1332,7 +1443,7 @@
                                  </div>
                                  <div id="songkick_filteroptions">
                                    <div class="row">
-                                      <label>Metro Area ID:</label>
+                                      <label>Metro Area ID</label>
                                      <input type="text" name="songkick_location" value="${config['songkick_location']}" size="10" title="Enter the Metro Area ID, e.g. the ID for London is 24426, this can be found by clicking the link and searching/selecting the city, e.g. London should find http://www.songkick.com/metro_areas/24426-uk-london">
                                        <a target="_blank" href="http://www.songkick.com/developer/location-search">Find Area ID</a>
                                    </div>
@@ -1345,9 +1456,9 @@
                                <div class="row">
                                    <label>Muscbrainz Mirror</label>
                                    <select name="mirror" id="mirror">
-                                    %for mirror in config['mirror_list']:
+                                    %for mirror in config['mirrorlist']:
                                        <%
-                                            if mirror == headphones.MIRROR:
+                                            if mirror == headphones.CONFIG.MIRROR:
                                                selected = 'selected="selected"'
                                            else:
                                                selected = ''
@@ -1363,6 +1474,17 @@
                                    <div class="row">
                                        <label>Port</label><input type="text" name="customport" value="${config['customport']}" size="8">
                                    </div>
+                                    <div class="row checkbox">
+                                        <input type="checkbox" name="customauth" id="customauth" value="1" ${config['customauth']} /><label>Requires Authentication</label>
+                                    </div>
+                                    <div id="customauth_options">
+                                        <div class="row">
+                                            <label>Username</label><input type="text" class="customuser" name="customuser" value="${config['customuser']}" size="20">
+                                        </div>
+                                        <div class="row">
+                                            <label>Password</label><input type="password" class="custompass" name="custompass" value="${config['custompass']}" size="15"><br>
+                                        </div>
+                                    </div>
                                    <div class="row">
                                        <label>Sleep Interval</label><input type="text" name="customsleep" value="${config['customsleep']}" size="4">
                                    </div>
@@ -1374,7 +1496,7 @@
                                    </div>
                                    <div class="row">
                                        <label>Password</label><input type="password" class="hppass" name="hppass" value="${config['hppass']}" size="15"><br>
-                                        <a href="http://headphones.codeshy.com/vip" id="vipserver" target="_blank">Get an Account!</a>
+                                        <a href="https://headphones.codeshy.com/vip" id="vipserver" target="_blank">Get an Account!</a>
                                    </div>
                                </div>
                            </fieldset>
@@ -1473,6 +1595,25 @@
                    $('#api_key').val(data);
            });
        });
+                if ($("#customauth").is(":checked"))
+                        {
+                            $("#customauth_options").show();
+                        }
+                else
+                        {
+                            $("#customauth_options").hide();
+                        }
+
+                $("#customauth").click(function(){
+                        if ($("#customauth").is(":checked"))
+                        {
+                            $("#customauth_options").slideDown();
+                        }
+                        else
+                        {
+                            $("#customauth_options").slideUp();
+                        }
+                });
                if ($("#enable_https").is(":checked"))
                        {
                            $("#https_options").show();
@@ -1813,6 +1954,26 @@
                        }
                });

+               if ($("#email").is(":checked"))
+                        {
+                            $("#email_options").show();
+                        }
+                else
+                        {
+                            $("#email_options").hide();
+                        }
+
+                $("#email").click(function(){
+                        if ($("#email").is(":checked"))
+                        {
+                            $("#email_options").slideDown();
+                        }
+                        else
+                        {
+                            $("#email_options").slideUp();
+                        }
+                });
+
                if ($("#songkick").is(":checked"))
                        {
                            $("#songkickoptions").show();
@@ -2008,18 +2169,20 @@
                    $( "#tabs" ).tabs();
                });
                initActions();
-                initConfigCheckbox("#use_headphones_indexer");
-                initConfigCheckbox("#usenewznab");
-                initConfigCheckbox("#usenzbsorg");
-                initConfigCheckbox("#useomgwtfnzbs");
-				initConfigCheckbox("#usekat");
-                initConfigCheckbox("#usepiratebay");
-                initConfigCheckbox("#usemininova");
-                initConfigCheckbox("#usewaffles");
-                initConfigCheckbox("#userutracker");
-                initConfigCheckbox("#usewhatcd");
-                initConfigCheckbox("#useapi");
+                initConfigCheckbox("#headphones_indexer");
+                initConfigCheckbox("#use_newznab");
+                initConfigCheckbox("#use_nzbsorg");
+                initConfigCheckbox("#use_omgwtfnzbs");
+                initConfigCheckbox("#use_kat");
+                initConfigCheckbox("#use_piratebay");
+                initConfigCheckbox("#use_oldpiratebay");
+                initConfigCheckbox("#use_mininova");
+                initConfigCheckbox("#use_waffles");
+                initConfigCheckbox("#use_rutracker");
+                initConfigCheckbox("#use_whatcd");
+                initConfigCheckbox("#api_enabled");
                initConfigCheckbox("#enable_https");
+                initConfigCheckbox("#customauth");


                $('#twitterStep1').click(function () {
@@ -336,6 +336,10 @@ form .row label {
  padding-top: 7px;
  width: 175px;
 }
+form .row label.inline {
+  margin-right: 5px;
+  width: auto;
+}
 form .row input {
  margin-right: 5px;
 }
@@ -1441,6 +1445,7 @@ div#artistheader h2 a {
 #album_table td#albumname,
 #album_table td#reldate,
 #album_table td#type,
+#album_table td#score,
 #track_table td#duration,
 #upcoming_table td#select,
 #upcoming_table td#status,
@@ -1451,6 +1456,7 @@ div#artistheader h2 a {
 }
 #album_table td#status,
 #album_table td#bitrate,
+#album_table td#score,
 #album_table td#albumformat,
 #album_table td#wantlossless {
  font-size: 13px;
@@ -191,6 +191,11 @@ form {
            line-height: normal;
            padding-top: 7px;
            width: 175px;
+
+            &.inline {
+                margin-right: 5px;
+                width: auto;
+            }
        }
        input { margin-right: 5px; }
        input[type=text], input[type=password] {
@@ -10,9 +10,10 @@
 			<a id="menu_link_delete" href="#" onclick="doAjaxCall('clearhistory?type=all',$(this),'table')" data-success="All History cleared"><i class="fa fa-trash-o"></i> Clear All History</a>
 			<a id="menu_link_delete" href="#" onclick="doAjaxCall('clearhistory?type=Processed',$(this),'table')" data-success="All Processed cleared"><i class="fa fa-trash-o"></i> Clear Processed</a>
 			<a id="menu_link_delete" href="#" onclick="doAjaxCall('clearhistory?type=Unprocessed',$(this),'table')" data-success="All Unprocessed cleared"><i class="fa fa-trash-o"></i> Clear Unprocessed</a>
+			<a id="menu_link_delete" href="#" onclick="doAjaxCall('clearhistory?type=Frozen',$(this),'table')" data-success="All Frozen cleared"><i class="fa fa-trash-o"></i> Clear Frozen</a>
 			<a id="menu_link_delete" href="#" onclick="doAjaxCall('clearhistory?type=Snatched',$(this),'table')" data-success="All Snatched cleared"><i class="fa fa-trash-o"></i> Clear Snatched</a>
 		</div>
-	</div>	
+	</div>
 </%def>

 <%def name="body()">
@@ -31,30 +32,35 @@
 			</tr>
 		</thead>
 		<tbody>
-		%for item in history:
-			<%
-				if item['Status'] == 'Processed':
-					grade = 'A'
-				elif item['Status'] == 'Snatched':
-					grade = 'C'
-				elif item['Status'] == 'Unprocessed':
-					grade = 'X'
-				else:
-					grade = 'U'
-					
-				fileid = 'unknown'
-				if item['URL'].find('nzb') != -1:
-					fileid = 'nzb'
-				if item['URL'].find('torrent') != -1:
-					fileid = 'torrent'
-				if item['URL'].find('rutracker') != -1:
-					fileid = 'torrent'
-			%>
+        %for item in history:
+        <%
+            if item['Status'] == 'Processed':
+                grade = 'A'
+            elif item['Status'] == 'Snatched':
+                grade = 'C'
+            elif item['Status'] == 'Unprocessed':
+                grade = 'X'
+            elif item['Status'] == 'Frozen':
+                grade = 'X'
+            else:
+                grade = 'U'
+
+            fileid = 'unknown'
+            if item['URL'].find('nzb') != -1:
+                fileid = 'nzb'
+            if item['URL'].find('torrent') != -1:
+                fileid = 'torrent'
+            if item['URL'].find('rutracker') != -1:
+                fileid = 'torrent'
+
+            folder = 'Folder: ' + item['FolderName']
+
+        %>
 			<tr class="grade${grade}">
 				<td id="dateadded">${item['DateAdded']}</td>
 				<td id="filename">${cgi.escape(item['Title'], quote=True)} [<a href="${item['URL']}">${fileid}</a>]<a href="albumPage?AlbumID=${item['AlbumID']}">[album page]</a></td>
 				<td id="size">${helpers.bytes_to_mb(item['Size'])}</td>
-				<td id="status">${item['Status']}</td>
+				<td title="${folder}" id="status">${item['Status']}</td>
 				<td id="action">[<a href="#" onclick="doAjaxCall('queueAlbum?AlbumID=${item['AlbumID']}&redirect=history', $(this),'table')"  data-success="Retrying download of '${cgi.escape(item['Title'], quote=True)}'">retry</a>][<a href="#" onclick="doAjaxCall('queueAlbum?AlbumID=${item['AlbumID']}&new=True&redirect=history',$(this),'table')"  data-success="Looking for a new version of '${cgi.escape(item['Title'], quote=True)}'">new</a>]</td>
 				<td id="delete"><a href="#" onclick="doAjaxCall('clearhistory?date_added=${item['DateAdded']}&title=${cgi.escape(item['Title'], quote=True)}',$(this),'table')" data-success="${cgi.escape(item['Title'], quote=True)} cleared from history"><img src="interfaces/default/images/trashcan.png" height="18" width="18" id="trashcan" title="Clear this item from the history"></a>
 			</tr>
@@ -81,8 +87,11 @@
 				"sInfoFiltered":"(filtered from _MAX_ total items)"},
 			"iDisplayLength": 25,
 			"sPaginationType": "full_numbers",
-			"aaSorting": []
-
+			"aaSorting": [],
+			"fnDrawCallback": function (o) {
+                // Jump to top of page
+                $('html,body').scrollTop(0);
+            }
 		});
 		resetFilters("history");
 	}
@@ -10,7 +10,7 @@
 				<th id="albumart"></th>
 				<th id="name">Artist Name</th>
 				<th id="status">Status</th>
-				<th id="album">Latest Album</th>
+				<th id="album">Latest Release</th>
 				<th id="have">Have</th>
 			</tr>
 		</thead>
@@ -131,6 +131,10 @@
                    },
                "fnInitComplete": function(oSettings, json)
                {
+                },
+                "fnDrawCallback": function (o) {
+                    // Jump to top of page
+                    $('html,body').scrollTop(0);
                }
            });
            $('#artist_table').on("draw.dt", function () {
@@ -7,6 +7,7 @@
    <div id="subhead_container">
        <div id="subhead_menu">
            <a class="menu_link_edit" href="clearLogs"><i class="fa fa-trash-o"></i> Clear log</a>
+            <a class="menu_link_edit" href="toggleVerbose"><i class="fa fa-pencil"></i> Toggle Debug Log</a>
        </div>
    </div>
 </%def>
@@ -46,51 +47,50 @@
 <%def name="javascriptIncludes()">
 	<script src="js/libs/jquery.dataTables.min.js"></script>
 	<script>
-$(document).ready(function() {
-    initActions();
+        $(document).ready(function() {
+            initActions();

-	$('#log_table').dataTable( {
-		"bProcessing": true,
-        "bServerSide": true,
-		"sAjaxSource": 'getLog',
-        "sPaginationType": "full_numbers",
-        "aaSorting": [[0, 'desc']],
-        "iDisplayLength": 25,
-        "bStateSave": true,
-        "oLanguage": {
-                    "sSearch":"Filter:",
-					"sLengthMenu":"Show _MENU_ lines per page",
-					"sEmptyTable": "No log information available",
-					"sInfo":"Showing _START_ to _END_ of _TOTAL_ lines",
-					"sInfoEmpty":"Showing 0 to 0 of 0 lines",
-					"sInfoFiltered":"(filtered from _MAX_ total lines)"},
-        "fnRowCallback": function (nRow, aData, iDisplayIndex, iDisplayIndexFull) {
-                                if (aData[1] === "ERROR")
-                                {
-                                    $('td', nRow).closest('tr').addClass("gradeX");
-                                }
-                                else if (aData[1] === "WARNING")
-                                {
-                                    $('td', nRow).closest('tr').addClass("gradeW");
-                                }
-                                else
-                                {
-                                    $('td', nRow).closest('tr').addClass("gradeZ");
-                                }
+        	$('#log_table').dataTable( {
+        		"bProcessing": true,
+                "bServerSide": true,
+        		"sAjaxSource": 'getLog',
+                "sPaginationType": "full_numbers",
+                "aaSorting": [[0, 'desc']],
+                "iDisplayLength": 25,
+                "bStateSave": true,
+                "oLanguage": {
+                            "sSearch":"Filter:",
+        					"sLengthMenu":"Show _MENU_ lines per page",
+        					"sEmptyTable": "No log information available",
+        					"sInfo":"Showing _START_ to _END_ of _TOTAL_ lines",
+        					"sInfoEmpty":"Showing 0 to 0 of 0 lines",
+        					"sInfoFiltered":"(filtered from _MAX_ total lines)"},
+                "fnRowCallback": function (nRow, aData, iDisplayIndex, iDisplayIndexFull) {
+                    if (aData[1] === "ERROR") {
+                        $('td', nRow).closest('tr').addClass("gradeX");
+                    } else if (aData[1] === "WARNING") {
+                        $('td', nRow).closest('tr').addClass("gradeW");
+                    } else {
+                        $('td', nRow).closest('tr').addClass("gradeZ");
+                    }

-                                return nRow;
-                                },
-        "fnServerData": function ( sSource, aoData, fnCallback ) {
-			/* Add some extra data to the sender */
-			$.getJSON( sSource, aoData, function (json) { 
-				fnCallback(json)
-			} );
-		}
-	} );
-} );
+                    return nRow;
+                },
+                "fnDrawCallback": function (o) {
+                    // Jump to top of page
+                    $('html,body').scrollTop(0);
+                },
+                "fnServerData": function ( sSource, aoData, fnCallback ) {
+        			/* Add some extra data to the sender */
+        			$.getJSON(sSource, aoData, function (json) {
+        				fnCallback(json)
+        			});
+        		}
+        	});
+        });
 	</script>
 	<script>
-		var timer; 
+		var timer;
 		function setRefresh()
 		{
 			refreshrate = document.getElementById('refreshrate');
@@ -20,7 +20,7 @@
 					</div>
 				</div>
 			<a class="menu_link_edit" href="manageArtists"><i class="fa fa-pencil"></i> Manage Artists</a>
-			%if not headphones.ADD_ARTISTS:
+			%if not headphones.CONFIG.AUTO_ADD_ARTISTS:
 			<a class="menu_link_edit" href="manageNew"><i class="fa fa-pencil"></i> Manage New Artists</a>
 			%endif
 			<a class="menu_link_edit" href="manageUnmatched"><i class="fa fa-pencil"></i> Manage Unmatched</a>
@@ -53,18 +53,18 @@
 					<br/>
 					<div class="row">
 								<label for="">Path to directory</label>
-					%if headphones.MUSIC_DIR:
-						<input type="text" value="${headphones.MUSIC_DIR}" name="path" size="70" />
+					%if headphones.CONFIG.MUSIC_DIR:
+						<input type="text" value="${headphones.CONFIG.MUSIC_DIR}" name="path" size="70" />
 					%else:
 						<input type="text" value="Enter a Music Directory to scan" onfocus="if
 						(this.value==this.defaultValue) this.value='';" name="path" size="70" />
 					%endif
 					</div>
 					<div class="row checkbox"> 
-					<input type="checkbox" name="libraryscan" id="libraryscan" value="1" ${checked(headphones.LIBRARYSCAN)}><label>Automatically scan library</label>
+					<input type="checkbox" name="libraryscan" id="libraryscan" value="1" ${checked(headphones.CONFIG.LIBRARYSCAN)}><label>Automatically scan library</label>
 					</div>
 					<div class="row checkbox"> 
-					<input type="checkbox" name="autoadd" id="autoadd" value="1" ${checked(headphones.ADD_ARTISTS)}><label>Auto-add new artists</label>
+					<input type="checkbox" name="autoadd" id="autoadd" value="1" ${checked(headphones.CONFIG.AUTO_ADD_ARTISTS)}><label>Auto-add new artists</label>
 					</div>
 								
 				</fieldset>
@@ -83,8 +83,8 @@
 			<div class="row">
 				<label for="">Username</label>
 				<%
-					if headphones.LASTFM_USERNAME:
-						lastfmvalue = headphones.LASTFM_USERNAME
+					if headphones.CONFIG.LASTFM_USERNAME:
+						lastfmvalue = headphones.CONFIG.LASTFM_USERNAME
 					else:
 						lastfmvalue = ''
 				%>
@@ -137,28 +137,27 @@
 					        No empty artists found.
 					    %endif
 					</div>
-				<a href="#" onclick="doAjaxCall('forcePostProcess',$(this))" data-success="Post-Processor is being loaded" data-error="Error during Post-Processing"><i class="fa fa-wrench fa-fw"></i> Force Post-Process Albums in Download Folder</a>
+					<div id="post_process">
+					<a href="#" class="btnOpenDialog"><i class="fa fa-wrench fa-fw"></i> Force Post-Process Albums in Download Folder</a>
+					</div>
 			</div>
 		</fieldset>
-		<form action="forcePostProcess" method="GET">
-		            <fieldset>
-		            <div class="row">
-		                <label>Force Post-Process Albums in Alternate Folder</label>
-					    <input type="text" value="" name="dir" id="dir" size="50" /><input type="submit" />
-		            </div>
-		            </fieldset>
-		      
-		</form>
-		<form action="forcePostProcess" method="GET">
-		            <fieldset>
-		            <div class="row">
-		                <label>Post-Process Single Folder</label>
-					    <input type="text" value="" name="album_dir" id="album_dir" size="50" /><input type="submit" />
-		            </div>
-		            </fieldset>
-		      
-		</form>
-		
+		<fieldset>
+            <div class="row" id="post_process_alternate">
+                <label>Force Post-Process Albums in Alternate Folder</label>
+			    <input type="text" value="" name="dir" id="dir" size="50" />
+			    <input type="button" class="btnOpenDialog" value="Submit" />
+            </div>
+        </fieldset>
+
+        <fieldset>
+            <div class="row" id="post_process_single">
+                <label>Post-Process Single Folder</label>
+			    <input type="text" value="" name="album_dir" id="album_dir" size="50" />
+			    <input type="button" class="btnOpenDialog" value="Submit" />
+            </div>
+        </fieldset>
+
 	</div>


@@ -177,8 +176,7 @@
 		</fieldset>
 		
 	</div>
-
-
+	<div id="dialog-confirm"></div>

 </div>
 </%def>
@@ -187,6 +185,55 @@
 		function addScanAction() {
 			$('#autoadd').append('<input type="hidden" name="scan" value=1 />');
 		};
+
+		function fnOpenNormalDialog(id) {
+			if (id === "post_process"){
+				var url = "forcePostProcess?"
+			}
+			if (id === "post_process_alternate"){
+				var dir = $('#dir').val();
+				var url = "forcePostProcess?dir=" + dir + "&"
+			}
+			if (id === "post_process_single"){
+				var dir = $('#album_dir').val();
+				var url = "forcePostProcess?album_dir=" + dir + "&"
+			}
+			var t = $('<a data-success="Post-Processor is being loaded" data-error="Error during Post-Processing">');
+		    $("#dialog-confirm").html("Do you want to keep the original folder(s) after post-processing? If you click no, the folders still might be kept depending on your global settings");
+
+		    // Define the Dialog and its properties.
+		    $("#dialog-confirm").dialog({
+		        resizable: false,
+		        modal: true,
+		        title: "Keep original folder(s)?",
+		        height: 170,
+		        width: 400,
+		        buttons: {
+		            "Yes": function () {
+		                $(this).dialog('close');
+		                doAjaxCall(url + "keep_original_folder=True", t);
+		            },
+		                "No": function () {
+		                $(this).dialog('close');
+		                doAjaxCall(url + "keep_original_folder=False", t);
+		            }
+		        }
+		    });
+		}
+
+		$('.btnOpenDialog').click(function(e){
+			e.preventDefault();
+			var parentId = $(this).closest('div').prop('id');
+			fnOpenNormalDialog(parentId);
+		});
+
+		function callback(value) {
+		    if (value) {
+		        alert("Confirmed");
+		    } else {
+		        alert("Rejected");
+		    }
+		}
 		
 		function initThisPage() {
 			$('#manage_albums').click(function() {
@@ -112,40 +112,43 @@
 <%def name="javascriptIncludes()">
 	<script src="js/libs/jquery.dataTables.min.js"></script>
 	<script>
+		function initThisPage() {
+			$('#album_table').dataTable({
+				"bDestroy": true,
+				"aoColumns": [
+	                null,
+					null,
+					null,
+					null,
+					null,
+					null,
+					{ "sType": "title-numeric"},
+					null,
+					null
+				],
+				"aoColumnDefs": [
+				          { 'bSortable': false, 'aTargets': [ 0 ] }
+				],
+				"oLanguage": {
+					"sLengthMenu":"Show _MENU_ albums per page",
+					"sEmptyTable": "No album information available",
+					"sInfo":"Showing _TOTAL_ albums",
+					"sInfoEmpty":"Showing 0 to 0 of 0 albums",
+					"sInfoFiltered":"(filtered from _MAX_ total albums)",
+					"sSearch": ""},
+				"bPaginate": false,
+				"aaSorting": [[5, 'desc']],
+				"fnDrawCallback": function (o) {
+	                // Jump to top of page
+	                $('html,body').scrollTop(0);
+	            }

-	function initThisPage() {
-		$('#album_table').dataTable({
-			"bDestroy": true,
-			"aoColumns": [
-                null,
-				null,
-				null,
-				null,
-				null,
-				null,
-				{ "sType": "title-numeric"},
-				null,
-				null
-			],
-			"aoColumnDefs": [
-			          { 'bSortable': false, 'aTargets': [ 0 ] }
-			],				
-			"oLanguage": {
-				"sLengthMenu":"Show _MENU_ albums per page",
-				"sEmptyTable": "No album information available",
-				"sInfo":"Showing _TOTAL_ albums",
-				"sInfoEmpty":"Showing 0 to 0 of 0 albums",
-				"sInfoFiltered":"(filtered from _MAX_ total albums)",
-				"sSearch": ""},
-			"bPaginate": false,
-			"aaSorting": [[5, 'desc']]
-
+			});
+			resetFilters("albums");
+		}
+		$(document).ready(function() {
+			initThisPage();
 		});
-		resetFilters("albums");
-	}
-	$(document).ready(function() {
-		initThisPage();
-	});
 	</script>
 </%def>

@@ -86,31 +86,31 @@
 <%def name="javascriptIncludes()">
 	<script src="js/libs/jquery.dataTables.min.js"></script>
 	<script>
-	function initThisPage() {
-		$('#artist_table').dataTable({	
-			"bDestroy":true,		
-			"aoColumns": [
-				null,
-				null,
-				{ "sType": "title-string"},
-				null,
-				{ "sType": "title-string"},
-				null
-			],
-			"oLanguage": {
-				"sSearch" : "",
-				"sEmptyTable": " "
-			},
-			"bStateSave": true,
-			"bPaginate": false
+		function initThisPage() {
+			$('#artist_table').dataTable({
+				"bDestroy": true,
+				"aoColumns": [
+					null,
+					null,
+					{ "sType": "title-string"},
+					null,
+					{ "sType": "title-string"},
+					null
+				],
+				"oLanguage": {
+					"sSearch" : "",
+					"sEmptyTable": " "
+				},
+				"bStateSave": true,
+				"bPaginate": false
+			});
+			resetFilters("artists");
+		}
+		$(document).ready(function() {
+			initThisPage();
+		});
+		$(window).load(function(){
+			initFancybox();
 		});
-		resetFilters("artists");
-	}
-	$(document).ready(function() {
-		initThisPage();
-	});
-	$(window).load(function(){
-		initFancybox();
-	});
 	</script>
 </%def>
@@ -17,7 +17,7 @@
 <%def name="body()">
 	<div class="table_wrapper">
 	<div id="manageheader" class="title">
-		<h1 class="clearfix"><i class="fa fa-music"></i> Manage Manually Changed Albums</h1>
+		<h1 class="clearfix"><i class="fa fa-music"></i> Manage Manually Matched Albums</h1>
 	</div>

 	<table class="display" id="artist_table">
@@ -37,30 +37,30 @@
 				old_album_clean = album['AlbumTitle'].replace('&','%26').replace('+', '%2B').replace("'","%27")
 			%>
 			<td id="artist">${album['ArtistName']}<BR>
-				<button id="restore_artist${count_albums}" onClick="restore_Artist(this.id)">(<-) Restore Artist</button>
-				<div id="restore_artist_dialog${count_albums}" title="Restore Artist" style="display:none">
+				<button id="reset_artist${count_albums}" onClick="reset_Artist(this.id)">(<-) Reset Artist</button>
+				<div id="reset_artist_dialog${count_albums}" title="Reset Artist" style="display:none">
 					<table>
-					<tr><td>Are you sure you want to restore Local Artist: ${album['ArtistName']} to unmatched?</td></tr>
+					<tr><td>Are you sure you want to reset Local Artist: ${album['ArtistName']} to unmatched?</td></tr>
 					<tr><td align="right"><BR>
 					%if album['AlbumStatus'] == "Ignored":
-						<button href="#" onclick="doAjaxCall('markManual?action=unignoreArtist&existing_artist=${old_artist_clean}', $(this), 'page');" data-success="Successfully restored ${album['ArtistName']} to unmatched">Restore Artist</button>
+						<button href="#" onclick="doAjaxCall('markManual?action=unignoreArtist&existing_artist=${old_artist_clean}', $(this), 'page');" data-success="Successfully reset ${album['ArtistName']} to unmatched">Reset Artist</button>
 					%elif album['AlbumStatus'] == "Matched":
-						<button href="#" onclick="doAjaxCall('markManual?action=unmatchArtist&existing_artist=${old_artist_clean}', $(this), 'page');" data-success="Successfully restored ${album['ArtistName']} to unmatched">Restore Artist</button>
+						<button href="#" onclick="doAjaxCall('markManual?action=unmatchArtist&existing_artist=${old_artist_clean}', $(this), 'page');" data-success="Successfully restored ${album['ArtistName']} to unmatched">Reset Artist</button>
 					%endif
 					</td></tr>
 					</table>
 				</div>
 			</td>
 			<td id="album">${album['AlbumTitle']}<BR>
-				<button id="restore_album${count_albums}" onClick="restore_Album(this.id)">(<-) Restore Album</button>
-				<div id="restore_album_dialog${count_albums}" title="Restore Album" style="display:none">
+				<button id="reset_album${count_albums}" onClick="reset_Album(this.id)">(<-) Reset Album</button>
+				<div id="reset_album_dialog${count_albums}" title="Reset Album" style="display:none">
 					<table>
-					<tr><td>Are you sure you want to restore Local Album: ${album['AlbumTitle']} to unmatched?</td></tr>
+					<tr><td>Are you sure you want to reset Local Album: ${album['AlbumTitle']} to unmatched?</td></tr>
 					<tr><td align="right"><BR>
 					%if album['AlbumStatus'] == "Ignored":
-						<button href="#" onclick="doAjaxCall('markManual?action=unignoreAlbum&existing_artist=${old_artist_clean}&existing_album=${old_album_clean}', $(this), 'page');" data-success="Successfully restored ${album['AlbumTitle']} to unmatched">Restore Album</button>
+						<button href="#" onclick="doAjaxCall('markManual?action=unignoreAlbum&existing_artist=${old_artist_clean}&existing_album=${old_album_clean}', $(this), 'page');" data-success="Successfully reset ${album['AlbumTitle']} to unmatched">Reset Album</button>
 					%elif album['AlbumStatus'] == "Matched":
-						<button href="#" onclick="doAjaxCall('markManual?action=unmatchAlbum&existing_artist=${old_artist_clean}&existing_album=${old_album_clean}', $(this), 'page');" data-success="Successfully restored ${album['AlbumTitle']} to unmatched">Restore Album</button>
+						<button href="#" onclick="doAjaxCall('markManual?action=unmatchAlbum&existing_artist=${old_artist_clean}&existing_album=${old_album_clean}', $(this), 'page');" data-success="Successfully reset ${album['AlbumTitle']} to unmatched">Reset Album</button>
 					%endif
 					</td></tr>
 					</table>
@@ -85,35 +85,37 @@
 <%def name="javascriptIncludes()">
 	<script src="js/libs/jquery.dataTables.min.js"></script>
 	<script>
-	$(document).ready(function()
-	{
-		$('#artist_table').dataTable(
-			{
-				"bStateSave": true,
-				"bPaginate": true,
-				"oLanguage": {
-					"sSearch": "",
-	                "sLengthMenu":"Show _MENU_ albums per page",
-						"sInfo":"Showing _START_ to _END_ of _TOTAL_ albums",
-						"sInfoEmpty":"Showing 0 to 0 of 0 albums",
-						"sInfoFiltered":"(filtered from _MAX_ total albums)",
-						"sEmptyTable": " ",
-	                    },
-	            "sPaginationType": "full_numbers",
-	         });
+	$(document).ready(function() {
+		$('#artist_table').dataTable({
+			"bStateSave": true,
+			"bPaginate": true,
+			"oLanguage": {
+				"sSearch": "",
+                "sLengthMenu":"Show _MENU_ albums per page",
+					"sInfo":"Showing _START_ to _END_ of _TOTAL_ albums",
+					"sInfoEmpty":"Showing 0 to 0 of 0 albums",
+					"sInfoFiltered":"(filtered from _MAX_ total albums)",
+					"sEmptyTable": " ",
+                    },
+            "sPaginationType": "full_numbers",
+            "fnDrawCallback": function (o) {
+                // Jump to top of page
+                $('html,body').scrollTop(0);
+            }
+         });

-initActions();
+		initActions();
 	});

-	function restore_Artist(clicked_id) {
-		n=clicked_id.replace("restore_artist","");
-		$("#restore_artist_dialog"+n).dialog();
+	function reset_Artist(clicked_id) {
+		n=clicked_id.replace("reset_artist","");
+		$("#reset_artist_dialog"+n).dialog();
 		return false;
 	}

-	function restore_Album(clicked_id) {
-		n=clicked_id.replace("restore_album","");
-		$("#restore_album_dialog"+n).dialog();
+	function reset_Album(clicked_id) {
+		n=clicked_id.replace("reset_album","");
+		$("#reset_album_dialog"+n).dialog();
 		return false;
 	}

@@ -6,7 +6,7 @@
 <%def name="headerIncludes()">
 	<div id="subhead_container">
 		<div id="subhead_menu">
-			<a id="menu_link_scan" onclick="doAjaxCall('musicScan?path=${headphones.MUSIC_DIR}&redirect=manageNew',$(this))" data-success="Music library is getting scanned">Scan Music Library</a>
+			<a id="menu_link_scan" onclick="doAjaxCall('musicScan?path=${headphones.CONFIG.MUSIC_DIR}&redirect=manageNew',$(this))" data-success="Music library is getting scanned">Scan Music Library</a>
 		</div>		
 	</div>	
 	<a href="manage" class="back">&laquo; Back to manage overview</a>
@@ -53,18 +53,16 @@
 <%def name="javascriptIncludes()">
 	<script src="js/libs/jquery.dataTables.min.js"></script>
 	<script>
-	$(document).ready(function()
-	{
-		$('#artist_table').dataTable(
-			{
-				"aaSorting": [[1, 'asc']],
-				"bStateSave": false,
-				"bPaginate": false,
-				"oLanguage": {
-					"sSearch" : ""},
-				
-			});
-initActions();
+	$(document).ready(function() {
+		$('#artist_table').dataTable({
+			"aaSorting": [[1, 'asc']],
+			"bStateSave": false,
+			"bPaginate": false,
+			"oLanguage": {
+				"sSearch" : ""},
+		});
+
+		initActions();
 	});
 	</script>
 </%def>
@@ -118,22 +118,24 @@
 <%def name="javascriptIncludes()">
 	<script src="js/libs/jquery.dataTables.min.js"></script>
 	<script>
-	$(document).ready(function()
-	{
-		$('#artist_table').dataTable(
-			{
-				"bStateSave": true,
-				"bPaginate": true,
-				"oLanguage": {
-					"sSearch": "",
-	                "sLengthMenu":"Show _MENU_ albums per page",
-						"sInfo":"Showing _START_ to _END_ of _TOTAL_ albums",
-						"sInfoEmpty":"Showing 0 to 0 of 0 albums",
-						"sInfoFiltered":"(filtered from _MAX_ total albums)",
-						"sEmptyTable": " ",
-	                    },
-	            "sPaginationType": "full_numbers",
-	         });
+	$(document).ready(function() {
+		$('#artist_table').dataTable({
+			"bStateSave": true,
+			"bPaginate": true,
+			"oLanguage": {
+				"sSearch": "",
+                "sLengthMenu":"Show _MENU_ albums per page",
+					"sInfo":"Showing _START_ to _END_ of _TOTAL_ albums",
+					"sInfoEmpty":"Showing 0 to 0 of 0 albums",
+					"sInfoFiltered":"(filtered from _MAX_ total albums)",
+					"sEmptyTable": " ",
+                    },
+            "sPaginationType": "full_numbers",
+            "fnDrawCallback": function (o) {
+                // Jump to top of page
+                $('html,body').scrollTop(0);
+            }
+         });

 initActions();
 	});
@@ -16,8 +16,13 @@
                    <th id="tracks">Tracks</th>
                    <th id="reldate">Date</th>
                    <th id="scoresmall">Score</th>
-                %else:
+                    <th id="mbrelid" style="display:none;"</th>
+                %elif type == 'artist':
                    <th id="artistname">Artist Name</th>
+                    <th id="score">Score</th>
+                %else:
+                    <th id="seriesname">Series Name</th>
+                    <th id="type">Type</th>
                    <th id="score">Score</th>
 				%endif
                <th id="mb"></th>
@@ -38,9 +43,11 @@
                %>
 				<tr class="grade${grade}">
 				%if type == 'album':
-					<td id="albumart" style=" text-align: center; vertical-align: middle;"><div id="artistImg"><img title="${result['albumid']}" class="albumArt" height="50" width="50" onerror="this.onerror=null;this.src='${caa_group_url}'"></div></td>
-                %else:
+					<td id="albumart" style=" text-align: center; vertical-align: middle;"><div id="artistImg"><img title="${result['albumid']}" class="albumArt" height="50" width="50" onerror="tryCCA(this, '${caa_group_url}')"></div></td>
+                %elif type == 'artist':
 					<td id="albumart"><div id="artistImg"><img title="${result['id']}" class="albumArt" height="50" width="50"></div></td>
+                %else:
+                    <td id="albumart"></td>
 				%endif
 				%if type == 'album':
 					<td id="albumname"><a href="addReleaseById?rid=${result['albumid']}&rgid=${result['rgid']}" title="${albuminfo}">${result['title']}</a></td>
@@ -50,9 +57,15 @@
                    <td id="reldate">${result['date']}</td>
                    <td id="score"><a href="${result['albumurl']} "title="View on MusicBrainz"><div class="bar"><div class="score" style="width: ${result['score']}px">${result['score']}</div></div></a></td>
                    <td id="musicbrainz" style=" text-align: center; line-height: 0; vertical-align: middle;"><a href="${result['albumurl']}"><img src="interfaces/default/images/MusicBrainz_Album_Icon.png" title="View on MusicBrainz" height="20" width="20"></a></td>
-                %else:
+                    <td id="mbrelid" style="display:none;">${result['albumid']}</td>
+                %elif type == 'artist':
                    <td id="artistname"><a href="addArtist?artistid=${result['id']}" title="${result['uniquename']}">${result['uniquename']}</a></td>
                    <td id="score"><a href="${result['url']} "title="View on MusicBrainz"><div class="bar"><div class="score" style="width: ${result['score']}px">${result['score']}</div></div></a></td>
+                    <td id="musicbrainz" style=" text-align: center; line-height: 0; vertical-align: middle;"><a href="${result['url']}"><img src="interfaces/default/images/MusicBrainz_Artist_Icon.png" title="View on MusicBrainz" height="20" width="20"></a></td>
+                %else:
+                    <td id="seriesname"><a href="addSeries?seriesid=${result['id']}" title="${result['uniquename']}">${result['uniquename']}</a></td>
+                    <td id="type">${result['type']}</td>
+                    <td id="score"><a href="${result['url']} "title="View on MusicBrainz"><div class="bar"><div class="score" style="width: ${result['score']}px">${result['score']}</div></div></a></td>
                    <td id="musicbrainz" style=" text-align: center; line-height: 0; vertical-align: middle;"><a href="${result['url']}"><img src="interfaces/default/images/MusicBrainz_Artist_Icon.png" title="View on MusicBrainz" height="20" width="20"></a></td>
 				%endif
 				</tr>
@@ -72,7 +85,13 @@
 	<script src="js/libs/jquery.dataTables.min.js"></script>

 	<script type="text/javascript">
-
+        function tryCCA(element, url) {
+            element.onerror = function() {
+                element.onerror = null;
+                element.src = "interfaces/default/images/no-cover-art.png";
+            };
+            element.src = url;
+        }
    	function getArt() {
    		$("table#searchresults_table tr td#albumart img").each(function(){
    			var id = $(this).attr('title');
@@ -85,7 +104,6 @@
    		});
    	}
    	function initThisPage() {
-            getArt();
    		$('#searchresults_table').dataTable({
 				"bDestroy": true,
 				"aoColumnDefs": [
@@ -100,11 +118,17 @@
 					"sSearch" : ""},
 				"iDisplayLength": 25,
 				"sPaginationType": "full_numbers",
-				"aaSorting": []
+				"aaSorting": [],
+                "fnDrawCallback": function (o) {
+                    // Jump to top of page
+                    $('html,body').scrollTop(0);
+                }
 			});
            $('#searchresults_table').on("draw.dt", function () {
+                getArt();
                $("img").unveil();
            });
+            getArt();
 			resetFilters("album");
    	}
    	$(document).ready(function(){
@@ -112,4 +136,18 @@
            initThisPage();
    	});
 	</script>
+
+    <script type="text/javascript">
+        <%!
+            # Abuse JSON module for escaping JavaScript
+            import json
+        %>
+        $(document).ready(function() {
+            // Search parameter
+            $("#searchbar input[name=name]").val(${name | json.dumps});
+
+            // Album or artist
+            $("#searchbar select[name=type]").val(${type | json.dumps});
+        });
+    </script>
 </%def>
@@ -97,7 +97,7 @@
 				"oLanguage": {
 					"sEmptyTable": " "
 				},
-				"bDestroy":true,
+				"bDestroy": true,
 				"bFilter": false,
 				"bInfo": false,
 				"bPaginate": false
@@ -13,15 +13,18 @@
 #  You should have received a copy of the GNU General Public License
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

-from headphones import request, db
+from headphones import request, db, logger
+

 def getAlbumArt(albumid):
    myDB = db.DBConnection()
-    asin = myDB.action('SELECT AlbumASIN from albums WHERE AlbumID=?', [albumid]).fetchone()[0]
+    asin = myDB.action(
+        'SELECT AlbumASIN from albums WHERE AlbumID=?', [albumid]).fetchone()[0]

    if asin:
        return 'http://ec1.images-amazon.com/images/P/%s.01.LZZZZZZZ.jpg' % asin

+
 def getCachedArt(albumid):
    from headphones import cache

@@ -14,69 +14,84 @@
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

 import headphones
-from headphones import db, logger
+from headphones import db, logger, cache
+

 def switch(AlbumID, ReleaseID):
-    '''
+    """
    Takes the contents from allalbums & alltracks (based on ReleaseID) and switches them into
    the albums & tracks table.
-    '''
+    """
+    logger.debug('Switching allalbums and alltracks')
    myDB = db.DBConnection()
-    oldalbumdata = myDB.action('SELECT * from albums WHERE AlbumID=?', [AlbumID]).fetchone()
-    newalbumdata = myDB.action('SELECT * from allalbums WHERE ReleaseID=?', [ReleaseID]).fetchone()
-    newtrackdata = myDB.action('SELECT * from alltracks WHERE ReleaseID=?', [ReleaseID]).fetchall()
+    oldalbumdata = myDB.action(
+        'SELECT * from albums WHERE AlbumID=?', [AlbumID]).fetchone()
+    newalbumdata = myDB.action(
+        'SELECT * from allalbums WHERE ReleaseID=?', [ReleaseID]).fetchone()
+    newtrackdata = myDB.action(
+        'SELECT * from alltracks WHERE ReleaseID=?', [ReleaseID]).fetchall()
    myDB.action('DELETE from tracks WHERE AlbumID=?', [AlbumID])

-    controlValueDict = {"AlbumID":  AlbumID}
+    controlValueDict = {"AlbumID": AlbumID}

-    newValueDict = {"ArtistID":         newalbumdata['ArtistID'],
-                    "ArtistName":       newalbumdata['ArtistName'],
-                    "AlbumTitle":       newalbumdata['AlbumTitle'],
-                    "ReleaseID":        newalbumdata['ReleaseID'],
-                    "AlbumASIN":        newalbumdata['AlbumASIN'],
-                    "ReleaseDate":      newalbumdata['ReleaseDate'],
-                    "Type":             newalbumdata['Type'],
-                    "ReleaseCountry":   newalbumdata['ReleaseCountry'],
-                    "ReleaseFormat":    newalbumdata['ReleaseFormat']
-                }
+    newValueDict = {"ArtistID": newalbumdata['ArtistID'],
+                    "ArtistName": newalbumdata['ArtistName'],
+                    "AlbumTitle": newalbumdata['AlbumTitle'],
+                    "ReleaseID": newalbumdata['ReleaseID'],
+                    "AlbumASIN": newalbumdata['AlbumASIN'],
+                    "ReleaseDate": newalbumdata['ReleaseDate'],
+                    "Type": newalbumdata['Type'],
+                    "ReleaseCountry": newalbumdata['ReleaseCountry'],
+                    "ReleaseFormat": newalbumdata['ReleaseFormat']
+                    }

    myDB.upsert("albums", newValueDict, controlValueDict)

+    # Update cache
+    c = cache.Cache()
+    c.remove_from_cache(AlbumID=AlbumID)
+    c.get_artwork_from_cache(AlbumID=AlbumID)
+
    for track in newtrackdata:

-        controlValueDict = {"TrackID":  track['TrackID'],
-                            "AlbumID":  AlbumID}
+        controlValueDict = {"TrackID": track['TrackID'],
+                            "AlbumID": AlbumID}

-        newValueDict = {"ArtistID":     track['ArtistID'],
-                    "ArtistName":       track['ArtistName'],
-                    "AlbumTitle":       track['AlbumTitle'],
-                    "AlbumASIN":        track['AlbumASIN'],
-                    "ReleaseID":        track['ReleaseID'],
-                    "TrackTitle":       track['TrackTitle'],
-                    "TrackDuration":    track['TrackDuration'],
-                    "TrackNumber":      track['TrackNumber'],
-                    "CleanName":        track['CleanName'],
-                    "Location":         track['Location'],
-                    "Format":           track['Format'],
-                    "BitRate":          track['BitRate']
-                    }
+        newValueDict = {"ArtistID": track['ArtistID'],
+                        "ArtistName": track['ArtistName'],
+                        "AlbumTitle": track['AlbumTitle'],
+                        "AlbumASIN": track['AlbumASIN'],
+                        "ReleaseID": track['ReleaseID'],
+                        "TrackTitle": track['TrackTitle'],
+                        "TrackDuration": track['TrackDuration'],
+                        "TrackNumber": track['TrackNumber'],
+                        "CleanName": track['CleanName'],
+                        "Location": track['Location'],
+                        "Format": track['Format'],
+                        "BitRate": track['BitRate']
+                        }

        myDB.upsert("tracks", newValueDict, controlValueDict)

-    # Mark albums as downloaded if they have at least 80% (by default, configurable) of the album
+    # Mark albums as downloaded if they have at least 80% (by default,
+    # configurable) of the album
    total_track_count = len(newtrackdata)
-    have_track_count = len(myDB.select('SELECT * from tracks WHERE AlbumID=? AND Location IS NOT NULL', [AlbumID]))
+    have_track_count = len(myDB.select(
+        'SELECT * from tracks WHERE AlbumID=? AND Location IS NOT NULL', [AlbumID]))

-    if oldalbumdata['Status'] == 'Skipped' and ((have_track_count/float(total_track_count)) >= (headphones.ALBUM_COMPLETION_PCT/100.0)):
-        myDB.action('UPDATE albums SET Status=? WHERE AlbumID=?', ['Downloaded', AlbumID])
+    if oldalbumdata['Status'] == 'Skipped' and ((have_track_count / float(total_track_count)) >= (headphones.CONFIG.ALBUM_COMPLETION_PCT / 100.0)):
+        myDB.action(
+            'UPDATE albums SET Status=? WHERE AlbumID=?', ['Downloaded', AlbumID])

    # Update have track counts on index
-    totaltracks = len(myDB.select('SELECT TrackTitle from tracks WHERE ArtistID=? AND AlbumID IN (SELECT AlbumID FROM albums WHERE Status != "Ignored")', [newalbumdata['ArtistID']]))
-    havetracks = len(myDB.select('SELECT TrackTitle from tracks WHERE ArtistID=? AND Location IS NOT NULL', [newalbumdata['ArtistID']]))
+    totaltracks = len(myDB.select(
+        'SELECT TrackTitle from tracks WHERE ArtistID=? AND AlbumID IN (SELECT AlbumID FROM albums WHERE Status != "Ignored")', [newalbumdata['ArtistID']]))
+    havetracks = len(myDB.select(
+        'SELECT TrackTitle from tracks WHERE ArtistID=? AND Location IS NOT NULL', [newalbumdata['ArtistID']]))

-    controlValueDict = {"ArtistID":     newalbumdata['ArtistID']}
+    controlValueDict = {"ArtistID": newalbumdata['ArtistID']}

-    newValueDict = {    "TotalTracks":      totaltracks,
-                        "HaveTracks":       havetracks}
+    newValueDict = {"TotalTracks": totaltracks,
+                    "HaveTracks": havetracks}

    myDB.upsert("artists", newValueDict, controlValueDict)
@@ -13,19 +13,18 @@
 #  You should have received a copy of the GNU General Public License
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

-from headphones import db, mb, importer, searcher, cache, postprocessor, versioncheck, logger
-
-from xml.dom.minidom import Document
+from headphones import db, mb, updater, importer, searcher, cache, postprocessor, versioncheck, logger

 import headphones
-import copy
 import json

-cmd_list = [ 'getIndex', 'getArtist', 'getAlbum', 'getUpcoming', 'getWanted', 'getSimilar', 'getHistory', 'getLogs',
+cmd_list = ['getIndex', 'getArtist', 'getAlbum', 'getUpcoming', 'getWanted', 'getSnatched', 'getSimilar', 'getHistory', 'getLogs',
            'findArtist', 'findAlbum', 'addArtist', 'delArtist', 'pauseArtist', 'resumeArtist', 'refreshArtist',
-            'addAlbum', 'queueAlbum', 'unqueueAlbum', 'forceSearch', 'forceProcess', 'getVersion', 'checkGithub',
-            'shutdown', 'restart', 'update', 'getArtistArt', 'getAlbumArt', 'getArtistInfo', 'getAlbumInfo',
-            'getArtistThumb', 'getAlbumThumb', 'choose_specific_download', 'download_specific_release']
+            'addAlbum', 'queueAlbum', 'unqueueAlbum', 'forceSearch', 'forceProcess', 'forceActiveArtistsUpdate', 
+            'getVersion', 'checkGithub', 'shutdown', 'restart', 'update', 'getArtistArt', 'getAlbumArt', 
+            'getArtistInfo', 'getAlbumInfo', 'getArtistThumb', 'getAlbumThumb', 'clearLogs',
+            'choose_specific_download', 'download_specific_release']
+

 class Api(object):

@@ -41,16 +40,15 @@ class Api(object):

        self.callback = None

+    def checkParams(self, *args, **kwargs):

-    def checkParams(self,*args,**kwargs):
-
-        if not headphones.API_ENABLED:
+        if not headphones.CONFIG.API_ENABLED:
            self.data = 'API not enabled'
            return
-        if not headphones.API_KEY:
+        if not headphones.CONFIG.API_KEY:
            self.data = 'API key not generated'
            return
-        if len(headphones.API_KEY) != 32:
+        if len(headphones.CONFIG.API_KEY) != 32:
            self.data = 'API key not generated correctly'
            return

@@ -58,7 +56,7 @@ class Api(object):
            self.data = 'Missing api key'
            return

-        if kwargs['apikey'] != headphones.API_KEY:
+        if kwargs['apikey'] != headphones.CONFIG.API_KEY:
            self.data = 'Incorrect API key'
            return
        else:
@@ -82,9 +80,9 @@ class Api(object):
        if self.data == 'OK':
            logger.info('Recieved API command: %s', self.cmd)
            methodToCall = getattr(self, "_" + self.cmd)
-            result = methodToCall(**self.kwargs)
+            methodToCall(**self.kwargs)
            if 'callback' not in self.kwargs:
-                if type(self.data) == type(''):
+                if isinstance(self.data, basestring):
                    return self.data
                else:
                    return json.dumps(self.data)
@@ -96,7 +94,7 @@ class Api(object):
        else:
            return self.data

-    def _dic_from_query(self,query):
+    def _dic_from_query(self, query):

        myDB = db.DBConnection()
        rows = myDB.select(query)
@@ -111,7 +109,8 @@ class Api(object):

    def _getIndex(self, **kwargs):

-        self.data = self._dic_from_query('SELECT * from artists order by ArtistSortName COLLATE NOCASE')
+        self.data = self._dic_from_query(
+            'SELECT * from artists order by ArtistSortName COLLATE NOCASE')
        return

    def _getArtist(self, **kwargs):
@@ -122,11 +121,15 @@ class Api(object):
        else:
            self.id = kwargs['id']

-        artist = self._dic_from_query('SELECT * from artists WHERE ArtistID="' + self.id + '"')
-        albums = self._dic_from_query('SELECT * from albums WHERE ArtistID="' + self.id + '" order by ReleaseDate DESC')
-        description = self._dic_from_query('SELECT * from descriptions WHERE ArtistID="' + self.id + '"')
+        artist = self._dic_from_query(
+            'SELECT * from artists WHERE ArtistID="' + self.id + '"')
+        albums = self._dic_from_query(
+            'SELECT * from albums WHERE ArtistID="' + self.id + '" order by ReleaseDate DESC')
+        description = self._dic_from_query(
+            'SELECT * from descriptions WHERE ArtistID="' + self.id + '"')

-        self.data = { 'artist': artist, 'albums': albums, 'description' : description }
+        self.data = {
+            'artist': artist, 'albums': albums, 'description': description}
        return

    def _getAlbum(self, **kwargs):
@@ -137,31 +140,49 @@ class Api(object):
        else:
            self.id = kwargs['id']

-        album = self._dic_from_query('SELECT * from albums WHERE AlbumID="' + self.id + '"')
-        tracks = self._dic_from_query('SELECT * from tracks WHERE AlbumID="' + self.id + '"')
-        description = self._dic_from_query('SELECT * from descriptions WHERE ReleaseGroupID="' + self.id + '"')
+        album = self._dic_from_query(
+            'SELECT * from albums WHERE AlbumID="' + self.id + '"')
+        tracks = self._dic_from_query(
+            'SELECT * from tracks WHERE AlbumID="' + self.id + '"')
+        description = self._dic_from_query(
+            'SELECT * from descriptions WHERE ReleaseGroupID="' + self.id + '"')

-        self.data = { 'album' : album, 'tracks' : tracks, 'description' : description }
+        self.data = {
+            'album': album, 'tracks': tracks, 'description': description}
        return

    def _getHistory(self, **kwargs):
-        self.data = self._dic_from_query('SELECT * from snatched WHERE status NOT LIKE "Seed%" order by DateAdded DESC')
+        self.data = self._dic_from_query(
+            'SELECT * from snatched WHERE status NOT LIKE "Seed%" order by DateAdded DESC')
        return

    def _getUpcoming(self, **kwargs):
-        self.data = self._dic_from_query("SELECT * from albums WHERE ReleaseDate > date('now') order by ReleaseDate DESC")
+        self.data = self._dic_from_query(
+            "SELECT * from albums WHERE ReleaseDate > date('now') order by ReleaseDate DESC")
        return

    def _getWanted(self, **kwargs):
-        self.data = self._dic_from_query("SELECT * from albums WHERE Status='Wanted'")
+        self.data = self._dic_from_query(
+            "SELECT * from albums WHERE Status='Wanted'")
        return

+    def _getSnatched(self, **kwargs):
+        self.data = self._dic_from_query(
+            "SELECT * from albums WHERE Status='Snatched'")
+        return
+        
    def _getSimilar(self, **kwargs):
        self.data = self._dic_from_query('SELECT * from lastfmcloud')
        return

    def _getLogs(self, **kwargs):
-        pass
+        self.data = headphones.LOG_LIST
+        return
+
+    def _clearLogs(self, **kwargs):
+        headphones.LOG_LIST = []
+        self.data = 'Cleared log'
+        return

    def _findArtist(self, **kwargs):
        if 'name' not in kwargs:
@@ -170,7 +191,7 @@ class Api(object):
        if 'limit' in kwargs:
            limit = kwargs['limit']
        else:
-            limit=50
+            limit = 50

        self.data = mb.findArtist(kwargs['name'], limit)

@@ -181,7 +202,7 @@ class Api(object):
        if 'limit' in kwargs:
            limit = kwargs['limit']
        else:
-            limit=50
+            limit = 50

        self.data = mb.findRelease(kwargs['name'], limit)

@@ -194,7 +215,7 @@ class Api(object):

        try:
            importer.addArtisttoDB(self.id)
-        except Exception, e:
+        except Exception as e:
            self.data = e

        return
@@ -244,7 +265,7 @@ class Api(object):

        try:
            importer.addArtisttoDB(self.id)
-        except Exception, e:
+        except Exception as e:
            self.data = e

        return
@@ -258,7 +279,7 @@ class Api(object):

        try:
            importer.addReleaseById(self.id)
-        except Exception, e:
+        except Exception as e:
            self.data = e

        return
@@ -312,13 +333,16 @@ class Api(object):
            self.dir = kwargs['dir']
        postprocessor.forcePostProcess(self.dir)

+    def _forceActiveArtistsUpdate(self, **kwargs):
+        updater.dbUpdate()
+
    def _getVersion(self, **kwargs):
        self.data = {
-            'git_path' : headphones.GIT_PATH,
-            'install_type' : headphones.INSTALL_TYPE,
-            'current_version' : headphones.CURRENT_VERSION,
-            'latest_version' : headphones.LATEST_VERSION,
-            'commits_behind' : headphones.COMMITS_BEHIND,
+            'git_path': headphones.CONFIG.GIT_PATH,
+            'install_type': headphones.INSTALL_TYPE,
+            'current_version': headphones.CURRENT_VERSION,
+            'latest_version': headphones.LATEST_VERSION,
+            'commits_behind': headphones.COMMITS_BEHIND,
        }

    def _checkGithub(self, **kwargs):
@@ -402,18 +426,19 @@ class Api(object):
        else:
            self.id = kwargs['id']

-        results = searcher.searchforalbum(self.id, choose_specific_download=True)
+        results = searcher.searchforalbum(
+            self.id, choose_specific_download=True)

        results_as_dicts = []

        for result in results:

            result_dict = {
-                'title':result[0],
-                'size':result[1],
-                'url':result[2],
-                'provider':result[3],
-                'kind':result[4]
+                'title': result[0],
+                'size': result[1],
+                'url': result[2],
+                'provider': result[3],
+                'kind': result[4]
            }
            results_as_dicts.append(result_dict)

@@ -421,7 +446,7 @@ class Api(object):

    def _download_specific_release(self, **kwargs):

-        expected_kwargs =['id', 'title','size','url','provider','kind']
+        expected_kwargs = ['id', 'title', 'size', 'url', 'provider', 'kind']

        for kwarg in expected_kwargs:
            if kwarg not in kwargs:
@@ -438,20 +463,24 @@ class Api(object):
        for kwarg in expected_kwargs:
            del kwargs[kwarg]

-        # Handle situations where the torrent url contains arguments that are parsed
+        # Handle situations where the torrent url contains arguments that are
+        # parsed
        if kwargs:
-            import urllib, urllib2
-            url = urllib2.quote(url, safe=":?/=&") + '&' + urllib.urlencode(kwargs)
+            import urllib
+            import urllib2
+            url = urllib2.quote(
+                url, safe=":?/=&") + '&' + urllib.urlencode(kwargs)

        try:
-            result = [(title,int(size),url,provider,kind)]
+            result = [(title, int(size), url, provider, kind)]
        except ValueError:
-            result = [(title,float(size),url,provider,kind)]
+            result = [(title, float(size), url, provider, kind)]

        logger.info(u"Making sure we can download the chosen result")
        (data, bestqual) = searcher.preprocess(result)

        if data and bestqual:
-          myDB = db.DBConnection()
-          album = myDB.action('SELECT * from albums WHERE AlbumID=?', [id]).fetchone()
-          searcher.send_to_downloader(data, bestqual, album)
+            myDB = db.DBConnection()
+            album = myDB.action(
+                'SELECT * from albums WHERE AlbumID=?', [id]).fetchone()
+            searcher.send_to_downloader(data, bestqual, album)
@@ -14,33 +14,34 @@
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

 import os
-import glob
-import urllib
 import headphones

 from headphones import db, helpers, logger, lastfm, request

 LASTFM_API_KEY = "690e1ed3bc00bc91804cd8f7fe5ed6d4"

+
 class Cache(object):
    """
-    This class deals with getting, storing and serving up artwork (album
-    art, artist images, etc) and info/descriptions (album info, artist descrptions)
-    to and from the cache folder. This can be called from within a web interface,
-    for example, using the helper functions getInfo(id) and getArtwork(id), to utilize the cached
-    images rather than having to retrieve them every time the page is reloaded.
+    This class deals with getting, storing and serving up artwork (album art,
+    artist images, etc) and info/descriptions (album info, artist descrptions)
+    to and from the cache folder. This can be called from within a web
+    interface. For example, using the helper functions `getInfo(id)` and
+    `getArtwork(id)`, to utilize the cached images rather than having to
+    retrieve them every time the page is reloaded.

-    So you can call cache.getArtwork(id) which will return an absolute path
-    to the image file on the local machine, or if the cache directory
-    doesn't exist, or can not be written to, it will return a url to the image.
+    You can call `getArtwork(id)` which will return an absolute path to the
+    image file on the local machine, or if the cache directory doesn't exist,
+    or can not be written to, it will return a url to the image.

-    Call cache.getInfo(id) to grab the artist/album info; will return the text description
+    Call `getInfo(id)` to grab the artist/album info. This will return the
+    text description.

-    The basic format for art in the cache is <musicbrainzid>.<date>.<ext>
-    and for info it is <musicbrainzid>.<date>.txt
+    The basic format for art in the cache is `<musicbrainzid>.<date>.<ext>`
+    and for info it is `<musicbrainzid>.<date>.txt`
    """

-    path_to_art_cache = os.path.join(headphones.CACHE_DIR, 'artwork')
+    path_to_art_cache = os.path.join(headphones.CONFIG.CACHE_DIR, 'artwork')

    def __init__(self):
        self.id = None
@@ -59,12 +60,12 @@ class Cache(object):
        self.info_summary = None
        self.info_content = None

-    def _findfilesstartingwith(self,pattern,folder):
+    def _findfilesstartingwith(self, pattern, folder):
        files = []
        if os.path.exists(folder):
            for fname in os.listdir(folder):
                if fname.startswith(pattern):
-                    files.append(os.path.join(folder,fname))
+                    files.append(os.path.join(folder, fname))
        return files

    def _exists(self, type):
@@ -72,14 +73,14 @@ class Cache(object):
        self.thumb_files = []

        if type == 'artwork':
-            self.artwork_files = self._findfilesstartingwith(self.id,self.path_to_art_cache)
+            self.artwork_files = self._findfilesstartingwith(self.id, self.path_to_art_cache)
            if self.artwork_files:
                return True
            else:
                return False

        elif type == 'thumb':
-            self.thumb_files = self._findfilesstartingwith("T_" + self.id,self.path_to_art_cache)
+            self.thumb_files = self._findfilesstartingwith("T_" + self.id, self.path_to_art_cache)
            if self.thumb_files:
                return True
            else:
@@ -88,11 +89,10 @@ class Cache(object):
    def _get_age(self, date):
        # There's probably a better way to do this
        split_date = date.split('-')
-        days_old = int(split_date[0])*365 + int(split_date[1])*30 + int(split_date[2])
+        days_old = int(split_date[0]) * 365 + int(split_date[1]) * 30 + int(split_date[2])

        return days_old

-
    def _is_current(self, filename=None, date=None):

        if filename:
@@ -191,18 +191,18 @@ class Cache(object):
        if not db_info or not db_info['LastUpdated'] or not self._is_current(date=db_info['LastUpdated']):

            self._update_cache()
-            info_dict = { 'Summary' : self.info_summary, 'Content' : self.info_content }
+            info_dict = {'Summary': self.info_summary, 'Content': self.info_content}
            return info_dict

        else:
-            info_dict = { 'Summary' : db_info['Summary'], 'Content' : db_info['Content'] }
+            info_dict = {'Summary': db_info['Summary'], 'Content': db_info['Content']}
            return info_dict

    def get_image_links(self, ArtistID=None, AlbumID=None):
-        '''
+        """
        Here we're just going to open up the last.fm url, grab the image links and return them
        Won't save any image urls, or save the artwork in the cache. Useful for search results, etc.
-        '''
+        """
        if ArtistID:

            self.id_type = 'artist'
@@ -240,15 +240,47 @@ class Cache(object):
            if not thumb_url:
                logger.debug('No album thumbnail image found on last.fm')

-        return {'artwork' : image_url, 'thumbnail' : thumb_url }
+        return {'artwork': image_url, 'thumbnail': thumb_url}
+
+    def remove_from_cache(self, ArtistID=None, AlbumID=None):
+        """
+        Pass a musicbrainz id to this function (either ArtistID or AlbumID)
+        """
+
+        if ArtistID:
+            self.id = ArtistID
+            self.id_type = 'artist'
+        else:
+            self.id = AlbumID
+            self.id_type = 'album'
+
+        self.query_type = 'artwork'
+
+        if self._exists('artwork'):
+            for artwork_file in self.artwork_files:
+                try:
+                    os.remove(artwork_file)
+                except:
+                    logger.warn('Error deleting file from the cache: %s', artwork_file)
+
+        self.query_type = 'thumb'
+
+        if self._exists('thumb'):
+            for thumb_file in self.thumb_files:
+                try:
+                    os.remove(thumb_file)
+                except Exception:
+                    logger.warn('Error deleting file from the cache: %s', thumb_file)

    def _update_cache(self):
-        '''
+        """
        Since we call the same url for both info and artwork, we'll update both at the same time
-        '''
+        """
+
        myDB = db.DBConnection()

        # Since lastfm uses release ids rather than release group ids for albums, we have to do a artist + album search for albums
+        # Exception is when adding albums manually, then we should use release id
        if self.id_type == 'artist':

            data = lastfm.request_lastfm("artist.getinfo", mbid=self.id, api_key=LASTFM_API_KEY)
@@ -277,9 +309,13 @@ class Cache(object):
                logger.debug('No artist thumbnail image found')

        else:
-
-            dbartist = myDB.action('SELECT ArtistName, AlbumTitle FROM albums WHERE AlbumID=?', [self.id]).fetchone()
-            data = lastfm.request_lastfm("album.getinfo", artist=dbartist['ArtistName'], album=dbartist['AlbumTitle'], api_key=LASTFM_API_KEY)
+            dbalbum = myDB.action('SELECT ArtistName, AlbumTitle, ReleaseID FROM albums WHERE AlbumID=?', [self.id]).fetchone()
+            if dbalbum['ReleaseID'] != self.id:
+                data = lastfm.request_lastfm("album.getinfo", mbid=dbalbum['ReleaseID'], api_key=LASTFM_API_KEY)
+                if not data:
+                    data = lastfm.request_lastfm("album.getinfo", artist=dbalbum['ArtistName'], album=dbalbum['AlbumTitle'], api_key=LASTFM_API_KEY)
+            else:
+                data = lastfm.request_lastfm("album.getinfo", artist=dbalbum['ArtistName'], album=dbalbum['AlbumTitle'], api_key=LASTFM_API_KEY)

            if not data:
                return
@@ -305,15 +341,16 @@ class Cache(object):
            if not thumb_url:
                logger.debug('No album thumbnail image found')

-        #Save the content & summary to the database no matter what if we've opened up the url
+        # Save the content & summary to the database no matter what if we've
+        # opened up the url
        if self.id_type == 'artist':
-            controlValueDict = {"ArtistID":     self.id}
+            controlValueDict = {"ArtistID": self.id}
        else:
-            controlValueDict = {"ReleaseGroupID":     self.id}
+            controlValueDict = {"ReleaseGroupID": self.id}

-        newValueDict = {"Summary":       self.info_summary,
-                        "Content":       self.info_content,
-                        "LastUpdated":   helpers.today()}
+        newValueDict = {"Summary": self.info_summary,
+                        "Content": self.info_content,
+                        "LastUpdated": helpers.today()}

        myDB.upsert("descriptions", newValueDict, controlValueDict)

@@ -331,7 +368,8 @@ class Cache(object):
            else:
                myDB.action('UPDATE albums SET ThumbURL=? WHERE AlbumID=?', [thumb_url, self.id])

-        # Should we grab the artwork here if we're just grabbing thumbs or info?? Probably not since the files can be quite big
+        # Should we grab the artwork here if we're just grabbing thumbs or
+        # info? Probably not since the files can be quite big
        if image_url and self.query_type == 'artwork':
            artwork = request.request_content(image_url, timeout=20)

@@ -340,12 +378,13 @@ class Cache(object):
                if not os.path.isdir(self.path_to_art_cache):
                    try:
                        os.makedirs(self.path_to_art_cache)
-                    except Exception, e:
+                        os.chmod(self.path_to_art_cache, int(headphones.CONFIG.FOLDER_PERMISSIONS, 8))
+                    except OSError as e:
                        logger.error('Unable to create artwork cache dir. Error: %s', e)
                        self.artwork_errors = True
                        self.artwork_url = image_url

-                #Delete the old stuff
+                # Delete the old stuff
                for artwork_file in self.artwork_files:
                    try:
                        os.remove(artwork_file)
@@ -358,13 +397,16 @@ class Cache(object):
                try:
                    with open(artwork_path, 'wb') as f:
                        f.write(artwork)
-                except IOError as e:
+
+                    os.chmod(artwork_path, int(headphones.CONFIG.FILE_PERMISSIONS, 8))
+                except (OSError, IOError) as e:
                    logger.error('Unable to write to the cache dir: %s', e)
                    self.artwork_errors = True
                    self.artwork_url = image_url

-        # Grab the thumbnail as well if we're getting the full artwork (as long as it's missing/outdated
-        if thumb_url and self.query_type in ['thumb','artwork'] and not (self.thumb_files and self._is_current(self.thumb_files[0])):
+        # Grab the thumbnail as well if we're getting the full artwork (as long
+        # as it's missing/outdated.
+        if thumb_url and self.query_type in ['thumb', 'artwork'] and not (self.thumb_files and self._is_current(self.thumb_files[0])):
            artwork = request.request_content(thumb_url, timeout=20)

            if artwork:
@@ -372,16 +414,17 @@ class Cache(object):
                if not os.path.isdir(self.path_to_art_cache):
                    try:
                        os.makedirs(self.path_to_art_cache)
-                    except Exception as e:
+                        os.chmod(self.path_to_art_cache, int(headphones.CONFIG.FOLDER_PERMISSIONS, 8))
+                    except OSError as e:
                        logger.error('Unable to create artwork cache dir. Error: %s' + e)
                        self.thumb_errors = True
                        self.thumb_url = thumb_url

-                #Delete the old stuff
+                # Delete the old stuff
                for thumb_file in self.thumb_files:
                    try:
                        os.remove(thumb_file)
-                    except Exception as e:
+                    except OSError as e:
                        logger.error('Error deleting file from the cache: %s', thumb_file)

                ext = os.path.splitext(image_url)[1]
@@ -390,13 +433,15 @@ class Cache(object):
                try:
                    with open(thumb_path, 'wb') as f:
                        f.write(artwork)
-                except IOError as e:
+
+                    os.chmod(thumb_path, int(headphones.CONFIG.FILE_PERMISSIONS, 8))
+                except (OSError, IOError) as e:
                    logger.error('Unable to write to the cache dir: %s', e)
                    self.thumb_errors = True
                    self.thumb_url = image_url

-def getArtwork(ArtistID=None, AlbumID=None):

+def getArtwork(ArtistID=None, AlbumID=None):
    c = Cache()
    artwork_path = c.get_artwork_from_cache(ArtistID, AlbumID)

@@ -409,8 +454,8 @@ def getArtwork(ArtistID=None, AlbumID=None):
        artwork_file = os.path.basename(artwork_path)
        return "cache/artwork/" + artwork_file

-def getThumb(ArtistID=None, AlbumID=None):

+def getThumb(ArtistID=None, AlbumID=None):
    c = Cache()
    artwork_path = c.get_thumb_from_cache(ArtistID, AlbumID)

@@ -423,16 +468,15 @@ def getThumb(ArtistID=None, AlbumID=None):
        thumbnail_file = os.path.basename(artwork_path)
        return "cache/artwork/" + thumbnail_file

+
 def getInfo(ArtistID=None, AlbumID=None):
-
    c = Cache()
-
    info_dict = c.get_info_from_cache(ArtistID, AlbumID)

    return info_dict

-def getImageLinks(ArtistID=None, AlbumID=None):

+def getImageLinks(ArtistID=None, AlbumID=None):
    c = Cache()
    image_links = c.get_image_links(ArtistID, AlbumID)

@@ -17,31 +17,32 @@
 ## Stolen from Sick-Beard's classes.py ##
 #########################################

-import headphones

 import urllib
-import datetime

 from common import USER_AGENT

+
 class HeadphonesURLopener(urllib.FancyURLopener):
    version = USER_AGENT

+
 class AuthURLOpener(HeadphonesURLopener):
    """
    URLOpener class that supports http auth without needing interactive password entry.
    If the provided username/password don't work it simply fails.
-    
+
    user: username to use for HTTP auth
    pw: password to use for HTTP auth
    """
+
    def __init__(self, user, pw):
        self.username = user
        self.password = pw

        # remember if we've tried the username/password before
        self.numTries = 0
-        
+
        # call the base class
        urllib.FancyURLopener.__init__(self)

@@ -55,7 +56,7 @@ class AuthURLOpener(HeadphonesURLopener):
        if self.numTries == 0:
            self.numTries = 1
            return (self.username, self.password)
-        
+
        # if we've tried before then return blank which cancels the request
        else:
            return ('', '')
@@ -65,6 +66,7 @@ class AuthURLOpener(HeadphonesURLopener):
        self.numTries = 0
        return HeadphonesURLopener.open(self, url)

+
 class SearchResult:
    """
    Represents a search result from an indexer.
@@ -87,7 +89,7 @@ class SearchResult:

    def __str__(self):

-        if self.provider == None:
+        if self.provider is None:
            return "Invalid provider, unable to print self"

        myString = self.provider.name + " @ " + self.url + "\n"
@@ -96,24 +98,28 @@ class SearchResult:
            myString += "  " + extra + "\n"
        return myString

+
 class NZBSearchResult(SearchResult):
    """
    Regular NZB result with an URL to the NZB
    """
    resultType = "nzb"

+
 class NZBDataSearchResult(SearchResult):
    """
    NZB result where the actual NZB XML data is stored in the extraInfo
    """
    resultType = "nzbdata"

+
 class TorrentSearchResult(SearchResult):
    """
    Torrent result with an URL to the torrent
    """
    resultType = "torrent"

+
 class Proper:
    def __init__(self, name, url, date):
        self.name = name
@@ -127,4 +133,4 @@ class Proper:
        self.episode = -1

    def __str__(self):
-        return str(self.date)+" "+self.name+" "+str(self.season)+"x"+str(self.episode)+" of "+str(self.tvdbid)
+        return str(self.date) + " " + self.name + " " + str(self.season) + "x" + str(self.episode) + " of " + str(self.tvdbid)
@@ -18,12 +18,15 @@ Created on Aug 1, 2011

@author: Michael
 '''
-import platform, operator, os, re
+import platform
+import operator
+import os
+import re

 from headphones import version

 #Identify Our Application
-USER_AGENT = 'Headphones/-'+version.HEADPHONES_VERSION+' ('+platform.system()+' '+platform.release()+')'
+USER_AGENT = 'Headphones/-' + version.HEADPHONES_VERSION + ' (' + platform.system() + ' ' + platform.release() + ')'

 ### Notification Types
 NOTIFY_SNATCH = 1
@@ -44,17 +47,18 @@ ARCHIVED = 6 # releases that you don't have locally (counts toward download comp
 IGNORED = 7 # releases that you don't want included in your download stats
 SNATCHED_PROPER = 9 # qualified with quality

+
 class Quality:

    NONE = 0
-    B192 = 1<<1     # 2
-    VBR =  1<<2     # 4
-    B256 = 1<<3     # 8
-    B320 = 1<<4     #16
-    FLAC = 1<<5     #32
+    B192 = 1 << 1     # 2
+    VBR = 1 << 2     # 4
+    B256 = 1 << 3     # 8
+    B320 = 1 << 4     #16
+    FLAC = 1 << 5     #32

    # put these bits at the other end of the spectrum, far enough out that they shouldn't interfere
-    UNKNOWN = 1<<15
+    UNKNOWN = 1 << 15

    qualityStrings = {NONE: "N/A",
                      UNKNOWN: "Unknown",
@@ -71,7 +75,7 @@ class Quality:
    def _getStatusStrings(status):
        toReturn = {}
        for x in Quality.qualityStrings.keys():
-            toReturn[Quality.compositeStatus(status, x)] = Quality.statusPrefixes[status]+" ("+Quality.qualityStrings[x]+")"
+            toReturn[Quality.compositeStatus(status, x)] = Quality.statusPrefixes[status] + " (" + Quality.qualityStrings[x] + ")"
        return toReturn

    @staticmethod
@@ -82,7 +86,7 @@ class Quality:
            anyQuality = reduce(operator.or_, anyQualities)
        if bestQualities:
            bestQuality = reduce(operator.or_, bestQualities)
-        return anyQuality | (bestQuality<<16)
+        return anyQuality | (bestQuality << 16)

    @staticmethod
    def splitQuality(quality):
@@ -91,7 +95,7 @@ class Quality:
        for curQual in Quality.qualityStrings.keys():
            if curQual & quality:
                anyQualities.append(curQual)
-            if curQual<<16 & quality:
+            if curQual << 16 & quality:
                bestQualities.append(curQual)

        return (anyQualities, bestQualities)
@@ -106,7 +110,7 @@ class Quality:
            if x == Quality.UNKNOWN:
                continue

-            regex = '\W'+Quality.qualityStrings[x].replace(' ','\W')+'\W'
+            regex = '\W' + Quality.qualityStrings[x].replace(' ', '\W') + '\W'
            regex_match = re.search(regex, name, re.I)
            if regex_match:
                return x
@@ -147,8 +151,8 @@ class Quality:
    def splitCompositeStatus(status):
        """Returns a tuple containing (status, quality)"""
        for x in sorted(Quality.qualityStrings.keys(), reverse=True):
-            if status > x*100:
-                return (status-x*100, x)
+            if status > x * 100:
+                return (status - x * 100, x)

        return (Quality.NONE, status)

@@ -0,0 +1,458 @@
+import headphones.logger
+import itertools
+import os
+import re
+from configobj import ConfigObj
+
+
+def bool_int(value):
+    """
+    Casts a config value into a 0 or 1
+    """
+    if isinstance(value, basestring):
+        if value.lower() in ('', '0', 'false', 'f', 'no', 'n', 'off'):
+            value = 0
+    return int(bool(value))
+
+_CONFIG_DEFINITIONS = {
+    'ADD_ALBUM_ART': (int, 'General', 0),
+    'ADVANCEDENCODER': (str, 'General', ''),
+    'ALBUM_ART_FORMAT': (str, 'General', 'folder'),
+    # This is used in importer.py to determine how complete an album needs to
+    # be - to be considered "downloaded". Percentage from 0-100
+    'ALBUM_COMPLETION_PCT': (int, 'Advanced', 80),
+    'API_ENABLED': (int, 'General', 0),
+    'API_KEY': (str, 'General', ''),
+    'AUTOWANT_ALL': (int, 'General', 0),
+    'AUTOWANT_MANUALLY_ADDED': (int, 'General', 1),
+    'AUTOWANT_UPCOMING': (int, 'General', 1),
+    'AUTO_ADD_ARTISTS': (int, 'General', 1),
+    'BITRATE': (int, 'General', 192),
+    'BLACKHOLE': (int, 'General', 0),
+    'BLACKHOLE_DIR': (str, 'General', ''),
+    'BOXCAR_ENABLED': (int, 'Boxcar', 0),
+    'BOXCAR_ONSNATCH': (int, 'Boxcar', 0),
+    'BOXCAR_TOKEN': (str, 'Boxcar', ''),
+    'CACHE_DIR': (str, 'General', ''),
+    'CACHE_SIZEMB': (int, 'Advanced', 32),
+    'CHECK_GITHUB': (int, 'General', 1),
+    'CHECK_GITHUB_INTERVAL': (int, 'General', 360),
+    'CHECK_GITHUB_ON_STARTUP': (int, 'General', 1),
+    'CLEANUP_FILES': (int, 'General', 0),
+    'CONFIG_VERSION': (str, 'General', '0'),
+    'CORRECT_METADATA': (int, 'General', 0),
+    'CUE_SPLIT': (int, 'General', 1),
+    'CUE_SPLIT_FLAC_PATH': (str, 'General', ''),
+    'CUE_SPLIT_SHNTOOL_PATH': (str, 'General', ''),
+    'CUSTOMAUTH': (int, 'General', 0),
+    'CUSTOMHOST': (str, 'General', 'localhost'),
+    'CUSTOMPASS': (str, 'General', ''),
+    'CUSTOMPORT': (int, 'General', 5000),
+    'CUSTOMSLEEP': (int, 'General', 1),
+    'CUSTOMUSER': (str, 'General', ''),
+    'DELETE_LOSSLESS_FILES': (int, 'General', 1),
+    'DESTINATION_DIR': (str, 'General', ''),
+    'DETECT_BITRATE': (int, 'General', 0),
+    'DOWNLOAD_DIR': (str, 'General', ''),
+    'DOWNLOAD_SCAN_INTERVAL': (int, 'General', 5),
+    'DOWNLOAD_TORRENT_DIR': (str, 'General', ''),
+    'DO_NOT_OVERRIDE_GIT_BRANCH': (int, 'General', 0),
+    'EMAIL_ENABLED': (int, 'Email', 0),
+    'EMAIL_FROM': (str, 'Email', ''),
+    'EMAIL_TO': (str, 'Email', ''),
+    'EMAIL_SMTP_SERVER': (str, 'Email', ''),
+    'EMAIL_SMTP_USER': (str, 'Email', ''),
+    'EMAIL_SMTP_PASSWORD': (str, 'Email', ''),
+    'EMAIL_SMTP_PORT': (int, 'Email', 25),
+    'EMAIL_SSL': (int, 'Email', 0),
+    'EMAIL_TLS': (int, 'Email', 0),
+    'EMAIL_ONSNATCH': (int, 'Email', 0),
+    'EMBED_ALBUM_ART': (int, 'General', 0),
+    'EMBED_LYRICS': (int, 'General', 0),
+    'ENABLE_HTTPS': (int, 'General', 0),
+    'ENCODER': (str, 'General', 'ffmpeg'),
+    'ENCODERFOLDER': (str, 'General', ''),
+    'ENCODERLOSSLESS': (int, 'General', 1),
+    'ENCODEROUTPUTFORMAT': (str, 'General', 'mp3'),
+    'ENCODERQUALITY': (int, 'General', 2),
+    'ENCODERVBRCBR': (str, 'General', 'cbr'),
+    'ENCODER_MULTICORE': (int, 'General', 0),
+    'ENCODER_MULTICORE_COUNT': (int, 'General', 0),
+    'ENCODER_PATH': (str, 'General', ''),
+    'EXTRAS': (str, 'General', ''),
+    'EXTRA_NEWZNABS': (list, 'Newznab', ''),
+    'FILE_FORMAT': (str, 'General', 'Track Artist - Album [Year] - Title'),
+    'FILE_PERMISSIONS': (str, 'General', '0644'),
+    'FILE_UNDERSCORES': (int, 'General', 0),
+    'FOLDER_FORMAT': (str, 'General', 'Artist/Album [Year]'),
+    'FOLDER_PERMISSIONS': (str, 'General', '0755'),
+    'FREEZE_DB': (int, 'General', 0),
+    'GIT_BRANCH': (str, 'General', 'master'),
+    'GIT_PATH': (str, 'General', ''),
+    'GIT_USER': (str, 'General', 'rembo10'),
+    'GROWL_ENABLED': (int, 'Growl', 0),
+    'GROWL_HOST': (str, 'Growl', ''),
+    'GROWL_ONSNATCH': (int, 'Growl', 0),
+    'GROWL_PASSWORD': (str, 'Growl', ''),
+    'HEADPHONES_INDEXER': (bool_int, 'General', False),
+    'HPPASS': (str, 'General', ''),
+    'HPUSER': (str, 'General', ''),
+    'HTTPS_CERT': (str, 'General', ''),
+    'HTTPS_KEY': (str, 'General', ''),
+    'HTTP_HOST': (str, 'General', '0.0.0.0'),
+    'HTTP_PASSWORD': (str, 'General', ''),
+    'HTTP_PORT': (int, 'General', 8181),
+    'HTTP_PROXY': (int, 'General', 0),
+    'HTTP_ROOT': (str, 'General', '/'),
+    'HTTP_USERNAME': (str, 'General', ''),
+    'IDTAG': (int, 'Beets', 0),
+    'IGNORE_CLEAN_RELEASES': (int, 'General', 0),
+    'IGNORED_WORDS': (str, 'General', ''),
+    'IGNORED_FOLDERS': (list, 'Advanced', []),
+    'IGNORED_FILES': (list, 'Advanced', []),
+    'INCLUDE_EXTRAS': (int, 'General', 0),
+    'INTERFACE': (str, 'General', 'default'),
+    'JOURNAL_MODE': (str, 'Advanced', 'wal'),
+    'KAT': (int, 'Kat', 0),
+    'KAT_PROXY_URL': (str, 'Kat', ''),
+    'KAT_RATIO': (str, 'Kat', ''),
+    'KEEP_NFO': (int, 'General', 0),
+    'KEEP_TORRENT_FILES': (int, 'General', 0),
+    'LASTFM_USERNAME': (str, 'General', ''),
+    'LAUNCH_BROWSER': (int, 'General', 1),
+    'LIBRARYSCAN': (int, 'General', 1),
+    'LIBRARYSCAN_INTERVAL': (int, 'General', 300),
+    'LMS_ENABLED': (int, 'LMS', 0),
+    'LMS_HOST': (str, 'LMS', ''),
+    'LOG_DIR': (str, 'General', ''),
+    'LOSSLESS_BITRATE_FROM': (int, 'General', 0),
+    'LOSSLESS_BITRATE_TO': (int, 'General', 0),
+    'LOSSLESS_DESTINATION_DIR': (str, 'General', ''),
+    'MB_IGNORE_AGE': (int, 'General', 365),
+    'MININOVA': (int, 'Mininova', 0),
+    'MININOVA_RATIO': (str, 'Mininova', ''),
+    'MIRROR': (str, 'General', 'musicbrainz.org'),
+    'MOVE_FILES': (int, 'General', 0),
+    'MPC_ENABLED': (bool_int, 'MPC', False),
+    'MUSIC_DIR': (str, 'General', ''),
+    'MUSIC_ENCODER': (int, 'General', 0),
+    'NEWZNAB': (int, 'Newznab', 0),
+    'NEWZNAB_APIKEY': (str, 'Newznab', ''),
+    'NEWZNAB_ENABLED': (int, 'Newznab', 1),
+    'NEWZNAB_HOST': (str, 'Newznab', ''),
+    'NMA_APIKEY': (str, 'NMA', ''),
+    'NMA_ENABLED': (int, 'NMA', 0),
+    'NMA_ONSNATCH': (int, 'NMA', 0),
+    'NMA_PRIORITY': (int, 'NMA', 0),
+    'NUMBEROFSEEDERS': (str, 'General', '10'),
+    'NZBGET_CATEGORY': (str, 'NZBget', ''),
+    'NZBGET_HOST': (str, 'NZBget', ''),
+    'NZBGET_PASSWORD': (str, 'NZBget', ''),
+    'NZBGET_PRIORITY': (int, 'NZBget', 0),
+    'NZBGET_USERNAME': (str, 'NZBget', 'nzbget'),
+    'NZBSORG': (int, 'NZBsorg', 0),
+    'NZBSORG_HASH': (str, 'NZBsorg', ''),
+    'NZBSORG_UID': (str, 'NZBsorg', ''),
+    'NZB_DOWNLOADER': (int, 'General', 0),
+    'OMGWTFNZBS': (int, 'omgwtfnzbs', 0),
+    'OMGWTFNZBS_APIKEY': (str, 'omgwtfnzbs', ''),
+    'OMGWTFNZBS_UID': (str, 'omgwtfnzbs', ''),
+    'OPEN_MAGNET_LINKS': (int, 'General', 0),  # 0: Ignore, 1: Open, 2: Convert
+    'MAGNET_LINKS': (int, 'General', 0),
+    'OSX_NOTIFY_APP': (str, 'OSX_Notify', '/Applications/Headphones'),
+    'OSX_NOTIFY_ENABLED': (int, 'OSX_Notify', 0),
+    'OSX_NOTIFY_ONSNATCH': (int, 'OSX_Notify', 0),
+    'PIRATEBAY': (int, 'Piratebay', 0),
+    'PIRATEBAY_PROXY_URL': (str, 'Piratebay', ''),
+    'PIRATEBAY_RATIO': (str, 'Piratebay', ''),
+    'OLDPIRATEBAY': (int, 'Old Piratebay', 0),
+    'OLDPIRATEBAY_URL': (str, 'Old Piratebay', ''),
+    'OLDPIRATEBAY_RATIO': (str, 'Old Piratebay', ''),
+    'PLEX_CLIENT_HOST': (str, 'Plex', ''),
+    'PLEX_ENABLED': (int, 'Plex', 0),
+    'PLEX_NOTIFY': (int, 'Plex', 0),
+    'PLEX_PASSWORD': (str, 'Plex', ''),
+    'PLEX_SERVER_HOST': (str, 'Plex', ''),
+    'PLEX_UPDATE': (int, 'Plex', 0),
+    'PLEX_USERNAME': (str, 'Plex', ''),
+    'PLEX_TOKEN': (str, 'Plex', ''),
+    'PREFERRED_BITRATE': (str, 'General', ''),
+    'PREFERRED_BITRATE_ALLOW_LOSSLESS': (int, 'General', 0),
+    'PREFERRED_BITRATE_HIGH_BUFFER': (int, 'General', 0),
+    'PREFERRED_BITRATE_LOW_BUFFER': (int, 'General', 0),
+    'PREFERRED_QUALITY': (int, 'General', 0),
+    'PREFERRED_WORDS': (str, 'General', ''),
+    'PREFER_TORRENTS': (int, 'General', 0),
+    'PROWL_ENABLED': (int, 'Prowl', 0),
+    'PROWL_KEYS': (str, 'Prowl', ''),
+    'PROWL_ONSNATCH': (int, 'Prowl', 0),
+    'PROWL_PRIORITY': (int, 'Prowl', 0),
+    'PUSHALOT_APIKEY': (str, 'Pushalot', ''),
+    'PUSHALOT_ENABLED': (int, 'Pushalot', 0),
+    'PUSHALOT_ONSNATCH': (int, 'Pushalot', 0),
+    'PUSHBULLET_APIKEY': (str, 'PushBullet', ''),
+    'PUSHBULLET_DEVICEID': (str, 'PushBullet', ''),
+    'PUSHBULLET_ENABLED': (int, 'PushBullet', 0),
+    'PUSHBULLET_ONSNATCH': (int, 'PushBullet', 0),
+    'PUSHOVER_APITOKEN': (str, 'Pushover', ''),
+    'PUSHOVER_ENABLED': (int, 'Pushover', 0),
+    'PUSHOVER_KEYS': (str, 'Pushover', ''),
+    'PUSHOVER_ONSNATCH': (int, 'Pushover', 0),
+    'PUSHOVER_PRIORITY': (int, 'Pushover', 0),
+    'RENAME_FILES': (int, 'General', 0),
+    'REPLACE_EXISTING_FOLDERS': (int, 'General', 0),
+    'KEEP_ORIGINAL_FOLDER': (int, 'General', 0),
+    'REQUIRED_WORDS': (str, 'General', ''),
+    'RUTRACKER': (int, 'Rutracker', 0),
+    'RUTRACKER_PASSWORD': (str, 'Rutracker', ''),
+    'RUTRACKER_RATIO': (str, 'Rutracker', ''),
+    'RUTRACKER_USER': (str, 'Rutracker', ''),
+    'SAB_APIKEY': (str, 'SABnzbd', ''),
+    'SAB_CATEGORY': (str, 'SABnzbd', ''),
+    'SAB_HOST': (str, 'SABnzbd', ''),
+    'SAB_PASSWORD': (str, 'SABnzbd', ''),
+    'SAB_USERNAME': (str, 'SABnzbd', ''),
+    'SAMPLINGFREQUENCY': (int, 'General', 44100),
+    'SEARCH_INTERVAL': (int, 'General', 1440),
+    'SONGKICK_APIKEY': (str, 'Songkick', 'nd1We7dFW2RqxPw8'),
+    'SONGKICK_ENABLED': (int, 'Songkick', 1),
+    'SONGKICK_FILTER_ENABLED': (int, 'Songkick', 0),
+    'SONGKICK_LOCATION': (str, 'Songkick', ''),
+    'SUBSONIC_ENABLED': (int, 'Subsonic', 0),
+    'SUBSONIC_HOST': (str, 'Subsonic', ''),
+    'SUBSONIC_PASSWORD': (str, 'Subsonic', ''),
+    'SUBSONIC_USERNAME': (str, 'Subsonic', ''),
+    'SYNOINDEX_ENABLED': (int, 'Synoindex', 0),
+    'TORRENTBLACKHOLE_DIR': (str, 'General', ''),
+    'TORRENT_DOWNLOADER': (int, 'General', 0),
+    'TORRENT_REMOVAL_INTERVAL': (int, 'General', 720),
+    'TRANSMISSION_HOST': (str, 'Transmission', ''),
+    'TRANSMISSION_PASSWORD': (str, 'Transmission', ''),
+    'TRANSMISSION_USERNAME': (str, 'Transmission', ''),
+    'TWITTER_ENABLED': (int, 'Twitter', 0),
+    'TWITTER_ONSNATCH': (int, 'Twitter', 0),
+    'TWITTER_PASSWORD': (str, 'Twitter', ''),
+    'TWITTER_PREFIX': (str, 'Twitter', 'Headphones'),
+    'TWITTER_USERNAME': (str, 'Twitter', ''),
+    'UPDATE_DB_INTERVAL': (int, 'General', 24),
+    'USENET_RETENTION': (int, 'General', '1500'),
+    'UTORRENT_HOST': (str, 'uTorrent', ''),
+    'UTORRENT_LABEL': (str, 'uTorrent', ''),
+    'UTORRENT_PASSWORD': (str, 'uTorrent', ''),
+    'UTORRENT_USERNAME': (str, 'uTorrent', ''),
+    'VERIFY_SSL_CERT': (bool_int, 'Advanced', 1),
+    'WAFFLES': (int, 'Waffles', 0),
+    'WAFFLES_PASSKEY': (str, 'Waffles', ''),
+    'WAFFLES_RATIO': (str, 'Waffles', ''),
+    'WAFFLES_UID': (str, 'Waffles', ''),
+    'WHATCD': (int, 'What.cd', 0),
+    'WHATCD_PASSWORD': (str, 'What.cd', ''),
+    'WHATCD_RATIO': (str, 'What.cd', ''),
+    'WHATCD_USERNAME': (str, 'What.cd', ''),
+    'XBMC_ENABLED': (int, 'XBMC', 0),
+    'XBMC_HOST': (str, 'XBMC', ''),
+    'XBMC_NOTIFY': (int, 'XBMC', 0),
+    'XBMC_PASSWORD': (str, 'XBMC', ''),
+    'XBMC_UPDATE': (int, 'XBMC', 0),
+    'XBMC_USERNAME': (str, 'XBMC', ''),
+    'XLDPROFILE': (str, 'General', '')
+}
+
+# pylint:disable=R0902
+# it might be nice to refactor for fewer instance variables
+class Config(object):
+    """ Wraps access to particular values in a config file """
+
+    def __init__(self, config_file):
+        """ Initialize the config with values from a file """
+        self._config_file = config_file
+        self._config = ConfigObj(self._config_file, encoding='utf-8')
+        for key in _CONFIG_DEFINITIONS.keys():
+            self.check_setting(key)
+        self.ENCODER_MULTICORE_COUNT = max(0, self.ENCODER_MULTICORE_COUNT)
+        self._upgrade()
+
+    def _define(self, name):
+        key = name.upper()
+        ini_key = name.lower()
+        definition = _CONFIG_DEFINITIONS[key]
+        if len(definition) == 3:
+            definition_type, section, default = definition
+        else:
+            definition_type, section, _, default = definition
+        return key, definition_type, section, ini_key, default
+
+    def check_section(self, section):
+        """ Check if INI section exists, if not create it """
+        if section not in self._config:
+            self._config[section] = {}
+            return True
+        else:
+            return False
+
+    def check_setting(self, key):
+        """ Cast any value in the config to the right type or use the default """
+        key, definition_type, section, ini_key, default = self._define(key)
+        self.check_section(section)
+        try:
+            my_val = definition_type(self._config[section][ini_key])
+        except Exception:
+            my_val = definition_type(default)
+            self._config[section][ini_key] = my_val
+        return my_val
+
+    def write(self):
+        """ Make a copy of the stored config and write it to the configured file """
+        new_config = ConfigObj(encoding="UTF-8")
+        new_config.filename = self._config_file
+
+        # first copy over everything from the old config, even if it is not
+        # correctly defined to keep from losing data
+        for key, subkeys in self._config.items():
+            if key not in new_config:
+                new_config[key] = {}
+            for subkey, value in subkeys.items():
+                new_config[key][subkey] = value
+
+        # next make sure that everything we expect to have defined is so
+        for key in _CONFIG_DEFINITIONS.keys():
+            key, definition_type, section, ini_key, default = self._define(key)
+            self.check_setting(key)
+            if section not in new_config:
+                new_config[section] = {}
+            new_config[section][ini_key] = self._config[section][ini_key]
+
+        # Write it to file
+        headphones.logger.info("Writing configuration to file")
+
+        try:
+            new_config.write()
+        except IOError as e:
+            headphones.logger.error("Error writing configuration file: %s", e)
+
+    def get_extra_newznabs(self):
+        """ Return the extra newznab tuples """
+        extra_newznabs = list(
+            itertools.izip(*[itertools.islice(self.EXTRA_NEWZNABS, i, None, 3)
+            for i in range(3)])
+        )
+        return extra_newznabs
+
+    def clear_extra_newznabs(self):
+        """ Forget about the configured extra newznabs """
+        self.EXTRA_NEWZNABS = []
+
+    def add_extra_newznab(self, newznab):
+        """ Add a new extra newznab """
+        extra_newznabs = self.EXTRA_NEWZNABS
+        for item in newznab:
+            extra_newznabs.append(item)
+        self.EXTRA_NEWZNABS = extra_newznabs
+
+    def __getattr__(self, name):
+        """
+        Returns something from the ini unless it is a real property
+        of the configuration object or is not all caps.
+        """
+        if not re.match(r'[A-Z_]+$', name):
+            return super(Config, self).__getattr__(name)
+        else:
+            return self.check_setting(name)
+
+    def __setattr__(self, name, value):
+        """
+        Maps all-caps properties to ini values unless they exist on the
+        configuration object.
+        """
+        if not re.match(r'[A-Z_]+$', name):
+            super(Config, self).__setattr__(name, value)
+            return value
+        else:
+            key, definition_type, section, ini_key, default = self._define(name)
+            self._config[section][ini_key] = definition_type(value)
+            return self._config[section][ini_key]
+
+    def process_kwargs(self, kwargs):
+        """
+        Given a big bunch of key value pairs, apply them to the ini.
+        """
+        for name, value in kwargs.items():
+            key, definition_type, section, ini_key, default = self._define(name)
+            self._config[section][ini_key] = definition_type(value)
+
+    def _upgrade(self):
+        """ Update folder formats in the config & bump up config version """
+        if self.CONFIG_VERSION == '0':
+            from headphones.helpers import replace_all
+            file_values = {
+                'tracknumber': 'Track',
+                'title': 'Title',
+                'artist': 'Artist',
+                'album': 'Album',
+                'year': 'Year'
+            }
+            folder_values = {
+                'artist': 'Artist',
+                'album': 'Album',
+                'year': 'Year',
+                'releasetype': 'Type',
+                'first': 'First',
+                'lowerfirst': 'first'
+            }
+            self.FILE_FORMAT = replace_all(self.FILE_FORMAT, file_values)
+            self.FOLDER_FORMAT = replace_all(self.FOLDER_FORMAT, folder_values)
+
+            self.CONFIG_VERSION = '1'
+
+        if self.CONFIG_VERSION == '1':
+            from headphones.helpers import replace_all
+            file_values = {
+                'Track': '$Track',
+                'Title': '$Title',
+                'Artist': '$Artist',
+                'Album': '$Album',
+                'Year': '$Year',
+                'track': '$track',
+                'title': '$title',
+                'artist': '$artist',
+                'album': '$album',
+                'year': '$year'
+            }
+            folder_values = {
+                'Artist': '$Artist',
+                'Album': '$Album',
+                'Year': '$Year',
+                'Type': '$Type',
+                'First': '$First',
+                'artist': '$artist',
+                'album': '$album',
+                'year': '$year',
+                'type': '$type',
+                'first': '$first'
+            }
+            self.FILE_FORMAT = replace_all(self.FILE_FORMAT, file_values)
+            self.FOLDER_FORMAT = replace_all(self.FOLDER_FORMAT, folder_values)
+            self.CONFIG_VERSION = '2'
+
+        if self.CONFIG_VERSION == '2':
+            # Update the config to use direct path to the encoder rather than the encoder folder
+            if self.ENCODERFOLDER:
+                self.ENCODER_PATH = os.path.join(self.ENCODERFOLDER, self.ENCODER)
+            self.CONFIG_VERSION = '3'
+
+        if self.CONFIG_VERSION == '3':
+            # Update the BLACKHOLE option to the NZB_DOWNLOADER format
+            if self.BLACKHOLE:
+                self.NZB_DOWNLOADER = 2
+            self.CONFIG_VERSION = '4'
+
+        # Enable Headphones Indexer if they have a VIP account
+        if self.CONFIG_VERSION == '4':
+            if self.HPUSER and self.HPPASS:
+                self.HEADPHONES_INDEXER = True
+            self.CONFIG_VERSION = '5'
+
+        if self.CONFIG_VERSION == '5':
+            if self.OPEN_MAGNET_LINKS:
+                self.MAGNET_LINKS = 2
+            self.CONFIG_VERSION = '5'
@@ -0,0 +1,666 @@
+#  This file is part of Headphones.
+#
+#  Headphones is free software: you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation, either version 3 of the License, or
+#  (at your option) any later version.
+#
+#  Headphones is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.
+
+# Most of this lifted from here: https://github.com/SzieberthAdam/gneposis-cdgrab
+
+import os
+import sys
+import re
+import subprocess
+import copy
+import glob
+
+import headphones
+from headphones import logger
+from mutagen.flac import FLAC
+
+CUE_HEADER = {
+    'genre': '^REM GENRE (.+?)$',
+    'date': '^REM DATE (.+?)$',
+    'discid': '^REM DISCID (.+?)$',
+    'comment': '^REM COMMENT (.+?)$',
+    'catalog': '^CATALOG (.+?)$',
+    'artist': '^PERFORMER (.+?)$',
+    'title': '^TITLE (.+?)$',
+    'file': '^FILE (.+?) (WAVE|FLAC)$',
+    'accurateripid': '^REM ACCURATERIPID (.+?)$'
+}
+
+CUE_TRACK = 'TRACK (\d\d) AUDIO$'
+
+CUE_TRACK_INFO = {
+    'artist': 'PERFORMER (.+?)$',
+    'title': 'TITLE (.+?)$',
+    'isrc': 'ISRC (.+?)$',
+    'index': 'INDEX (\d\d) (.+?)$'
+}
+
+ALBUM_META_FILE_NAME = 'album.dat'
+SPLIT_FILE_NAME = 'split.dat'
+
+ALBUM_META_ALBUM_BY_CUE = ('artist', 'title', 'date', 'genre')
+
+HTOA_LENGTH_TRIGGER = 3
+
+WAVE_FILE_TYPE_BY_EXTENSION = {
+    '.wav': 'Waveform Audio',
+    '.wv': 'WavPack',
+    '.ape': "Monkey's Audio",
+    '.m4a': 'Apple Lossless',
+    '.flac': 'Free Lossless Audio Codec'
+}
+
+#SHNTOOL_COMPATIBLE = ("Free Lossless Audio Codec", "Waveform Audio", "Monkey's Audio")
+
+# TODO: Make this better!
+# this module-level variable is bad. :(
+CUE_META = None
+
+
+def check_splitter(command):
+    '''Check xld or shntool installed'''
+    try:
+        env = os.environ.copy()
+        if 'xld' in command:
+            env['PATH'] += os.pathsep + '/Applications'
+        elif headphones.CONFIG.CUE_SPLIT_FLAC_PATH:
+            command = os.path.join(headphones.CONFIG.CUE_SPLIT_SHNTOOL_PATH, 'shntool')
+        devnull = open(os.devnull)
+        subprocess.Popen([command], stdout=devnull, stderr=devnull, env=env).communicate()
+    except OSError as e:
+        if e.errno == os.errno.ENOENT:
+            return False
+    return True
+
+
+def split_baby(split_file, split_cmd):
+    '''Let's split baby'''
+    logger.info('Splitting %s...', split_file.decode(headphones.SYS_ENCODING, 'replace'))
+    logger.debug(subprocess.list2cmdline(split_cmd))
+
+    # Prevent Windows from opening a terminal window
+    startupinfo = None
+
+    if headphones.SYS_PLATFORM == "win32":
+        startupinfo = subprocess.STARTUPINFO()
+        try:
+            startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
+        except AttributeError:
+            startupinfo.dwFlags |= subprocess._subprocess.STARTF_USESHOWWINDOW
+
+    env = os.environ.copy()
+    if 'xld' in split_cmd:
+        env['PATH'] += os.pathsep + '/Applications'
+    elif headphones.CONFIG.CUE_SPLIT_FLAC_PATH:
+        env['PATH'] += os.pathsep + headphones.CONFIG.CUE_SPLIT_FLAC_PATH
+
+    process = subprocess.Popen(split_cmd, startupinfo=startupinfo,
+                               stdin=open(os.devnull, 'rb'), stdout=subprocess.PIPE,
+                               stderr=subprocess.PIPE, env=env)
+    stdout, stderr = process.communicate()
+
+    if process.returncode:
+        logger.error('Split failed for %s', split_file.decode(headphones.SYS_ENCODING, 'replace'))
+        out = stdout if stdout else stderr
+        logger.error('Error details: %s', out.decode(headphones.SYS_ENCODING, 'replace'))
+        return False
+    else:
+        logger.info('Split success %s', split_file.decode(headphones.SYS_ENCODING, 'replace'))
+        return True
+
+
+def check_list(list, ignore=0):
+    '''Checks a list for None elements. If list have None (after ignore index) then it should pass only if all elements
+    are None threreafter. Returns a tuple without the None entries.'''
+
+    if ignore:
+        try:
+            list[int(ignore)]
+        except:
+            raise ValueError('non-integer ignore index or ignore index not in list')
+
+    list1 = list[:ignore]
+    list2 = list[ignore:]
+
+    try:
+        first_none = list2.index(None)
+    except:
+        return tuple(list1 + list2)
+
+    for i in range(first_none, len(list2)):
+        if list2[i]:
+            raise ValueError('non-None entry after None entry in list at index {0}'.format(i))
+
+    while True:
+        list2.remove(None)
+        try:
+            list2.index(None)
+        except:
+            break
+
+    return tuple(list1 + list2)
+
+
+def trim_cue_entry(string):
+    '''Removes leading and trailing "s.'''
+    if string[0] == '"' and string[-1] == '"':
+        string = string[1:-1]
+    return string
+
+
+def int_to_str(value, length=2):
+    '''Converts integer to string eg 3 to "03"'''
+    try:
+        int(value)
+    except:
+        raise ValueError('expected an integer value')
+
+    content = str(value)
+    while len(content) < length:
+        content = '0' + content
+    return content
+
+
+class Directory:
+    def __init__(self, path):
+        self.path = path
+        self.name = os.path.split(self.path)[-1]
+        self.content = []
+        self.update()
+
+    def filter(self, classname):
+        content = []
+        for c in self.content:
+            if c.__class__.__name__ == classname:
+                content.append(c)
+        return content
+
+    def tracks(self, ext=None, split=False):
+        content = []
+        for c in self.content:
+            ext_match = False
+            if c.__class__.__name__ == 'WaveFile':
+                if not ext or (ext and ext == c.name_ext):
+                    ext_match = True
+                if ext_match and c.track_nr:
+                    if not split or (split and c.split_file):
+                        content.append(c)
+        return content
+
+    def update(self):
+        def check_match(filename):
+            for i in self.content:
+                if i.name == filename:
+                    return True
+            return False
+
+        def identify_track_number(filename):
+            if 'split-track' in filename:
+                search = re.search('split-track(\d\d)', filename)
+                if search:
+                    n = int(search.group(1))
+                    if n:
+                        return n
+            for n in range(0, 100):
+                search = re.search(int_to_str(n), filename)
+                if search:
+                    # TODO: not part of other value such as year
+                    return n
+
+        list_dir = glob.glob1(self.path, '*')
+
+        # TODO: for some reason removes only one file
+        rem_list = []
+        for i in self.content:
+            if i.name not in list_dir:
+                rem_list.append(i)
+        for i in rem_list:
+            self.content.remove(i)
+
+        for i in list_dir:
+            if not check_match(i):
+                # music file
+                if os.path.splitext(i)[-1] in WAVE_FILE_TYPE_BY_EXTENSION.keys():
+                    track_nr = identify_track_number(i)
+                    if track_nr:
+                        self.content.append(WaveFile(self.path + os.sep + i, track_nr=track_nr))
+                    else:
+                        self.content.append(WaveFile(self.path + os.sep + i))
+
+                # cue file
+                elif os.path.splitext(i)[-1] == '.cue':
+                    self.content.append(CueFile(self.path + os.sep + i))
+
+                # meta file
+                elif i == ALBUM_META_FILE_NAME:
+                    self.content.append(MetaFile(self.path + os.sep + i))
+
+                # directory
+                elif os.path.isdir(i):
+                    self.content.append(Directory(self.path + os.sep + i))
+
+                else:
+                    self.content.append(File(self.path + os.sep + i))
+
+
+class File(object):
+    def __init__(self, path):
+        self.path = path
+        self.name = os.path.split(self.path)[-1]
+
+        self.name_name = ''.join(os.path.splitext(self.name)[:-1])
+        self.name_ext = os.path.splitext(self.name)[-1]
+        self.split_file = True if self.name_name[:11] == 'split-track' else False
+
+    def get_name(self, ext=True, cmd=False):
+
+        if ext is True:
+            content = self.name
+        elif ext is False:
+            content = self.name_name
+        elif ext[0] == '.':
+            content = self.name_name + ext
+        else:
+            raise ValueError('ext parameter error')
+
+        if cmd:
+            content = content.replace(' ', '\ ')
+
+        return content
+
+
+class CueFile(File):
+    def __init__(self, path):
+
+        def header_parser():
+            global line_content
+            c = self.content.splitlines()
+            header_dict = {}
+            #remaining_headers = CUE_HEADER
+            remaining_headers = copy.copy(CUE_HEADER)
+            line_index = 0
+            match = True
+            while match:
+                match = False
+                saved_match = None
+                line_content = c[line_index]
+                for e in remaining_headers:
+                    search_result = re.search(remaining_headers[e], line_content, re.I)
+                    if search_result:
+                        search_content = trim_cue_entry(search_result.group(1))
+                        header_dict[e] = search_content
+                        saved_match = e
+                        match = True
+                        line_index += 1
+                if saved_match:
+                    del remaining_headers[saved_match]
+            return header_dict, line_index
+
+        def track_parser(start_line):
+            c = self.content.splitlines()
+            line_index = start_line
+            line_content = c[line_index]
+            search_result = re.search(CUE_TRACK, line_content, re.I)
+            if not search_result:
+                raise ValueError('inconsistent CUE sheet, TRACK expected at line {0}'.format(line_index + 1))
+            track_nr = int(search_result.group(1))
+            line_index += 1
+            next_track = False
+            track_meta = {}
+            # we make room for future indexes
+            track_meta['index'] = [None for m in range(100)]
+
+            while not next_track:
+                if line_index < len(c):
+                    line_content = c[line_index]
+
+                    artist_search = re.search(CUE_TRACK_INFO['artist'], line_content, re.I)
+                    title_search = re.search(CUE_TRACK_INFO['title'], line_content, re.I)
+                    isrc_search = re.search(CUE_TRACK_INFO['isrc'], line_content, re.I)
+                    index_search = re.search(CUE_TRACK_INFO['index'], line_content, re.I)
+
+                    if artist_search:
+                        if trim_cue_entry(artist_search.group(1)) != self.header['artist']:
+                            track_meta['artist'] = trim_cue_entry(artist_search.group(1))
+                        line_index += 1
+                    elif title_search:
+                        track_meta['title'] = trim_cue_entry(title_search.group(1))
+                        line_index += 1
+                    elif isrc_search:
+                        track_meta['isrc'] = trim_cue_entry(isrc_search.group(1))
+                        line_index += 1
+                    elif index_search:
+                        track_meta['index'][int(index_search.group(1))] = index_search.group(2)
+                        line_index += 1
+                    elif re.search(CUE_TRACK, line_content, re.I):
+                        next_track = True
+                    elif line_index == len(c) - 1 and not line_content:
+                        # last line is empty
+                        line_index += 1
+                    elif re.search('FLAGS DCP$', line_content, re.I):
+                        track_meta['dcpflag'] = True
+                        line_index += 1
+                    else:
+                        raise ValueError('unknown entry in track error, line {0}'.format(line_index + 1))
+                else:
+                    next_track = True
+
+            track_meta['index'] = check_list(track_meta['index'], ignore=1)
+
+            return track_nr, track_meta, line_index
+
+        super(CueFile, self).__init__(path)
+
+        try:
+            with open(self.name) as cue_file:
+                self.content = cue_file.read()
+        except:
+            self.content = None
+
+        if not self.content:
+            try:
+                 with open(self.name, encoding="cp1252") as cue_file:
+                     self.content = cue_file.read()
+            except:
+                raise ValueError('Cant encode CUE Sheet.')
+
+        if self.content[0] == u'\ufeff':
+            self.content = self.content[1:]
+
+        header = header_parser()
+
+        self.header = header[0]
+
+        line_index = header[1]
+
+        # we make room for tracks
+        tracks = [None for m in range(100)]
+
+        while line_index < len(self.content.splitlines()):
+            parsed_track = track_parser(line_index)
+            line_index = parsed_track[2]
+            tracks[parsed_track[0]] = parsed_track[1]
+
+        self.tracks = check_list(tracks, ignore=1)
+
+    def get_meta(self):
+        content = ''
+        for i in ALBUM_META_ALBUM_BY_CUE:
+            if self.header.get(i):
+                content += i + '\t' + self.header[i] + '\n'
+            else:
+                content += i + '\t' + '\n'
+
+        for i in range(len(self.tracks)):
+            if self.tracks[i]:
+                if self.tracks[i].get('artist'):
+                    content += 'track' + int_to_str(i) + 'artist' + '\t' + self.tracks[i].get('artist') + '\n'
+                if self.tracks[i].get('title'):
+                    content += 'track' + int_to_str(i) + 'title' + '\t' + self.tracks[i].get('title') + '\n'
+        return content
+
+    def htoa(self):
+        '''Returns true if Hidden Track exists.'''
+        if int(self.tracks[1]['index'][1][-5:-3]) >= HTOA_LENGTH_TRIGGER:
+            return True
+        return False
+
+    def breakpoints(self):
+        '''Returns track break points. Identical as CUETools' cuebreakpoints, with the exception of my standards for HTOA.'''
+        content = ''
+        for t in range(len(self.tracks)):
+            if t == 1 and not self.htoa():
+                content += ''
+            elif t >= 1:
+                t_index = self.tracks[t]['index']
+                content += t_index[1]
+                if t < (len(self.tracks) - 1):
+                    content += '\n'
+        return content
+
+
+class MetaFile(File):
+    def __init__(self, path):
+        super(MetaFile, self).__init__(path)
+        with open(self.path) as meta_file:
+            self.rawcontent = meta_file.read()
+
+        content = {}
+        content['tracks'] = [None for m in range(100)]
+
+        for l in self.rawcontent.splitlines():
+            parsed_line = re.search('^(.+?)\t(.+?)$', l)
+            if parsed_line:
+                if parsed_line.group(1)[:5] == 'track':
+                    parsed_track = re.search('^track(\d\d)(.+?)$', parsed_line.group(1))
+                    if not parsed_track:
+                        raise ValueError('Syntax error in album meta file')
+                    if not content['tracks'][int(parsed_track.group(1))]:
+                        content['tracks'][int(parsed_track.group(1))] = dict()
+                    content['tracks'][int(parsed_track.group(1))][parsed_track.group(2)] = parsed_line.group(2)
+                else:
+                    content[parsed_line.group(1)] = parsed_line.group(2)
+
+        content['tracks'] = check_list(content['tracks'], ignore=1)
+
+        self.content = content
+
+    def flac_tags(self, track_nr):
+        common_tags = dict()
+        freeform_tags = dict()
+
+        # common flac tags
+        common_tags['artist'] = self.content['artist']
+        common_tags['album'] = self.content['title']
+        common_tags['title'] = self.content['tracks'][track_nr]['title']
+        common_tags['tracknumber'] = str(track_nr)
+        common_tags['tracktotal'] = str(len(self.content['tracks']) - 1)
+        if 'date' in self.content:
+            common_tags['date'] = self.content['date']
+        if 'genre' in CUE_META.content:
+            common_tags['genre'] = CUE_META.content['genre']
+
+        #freeform tags
+        #freeform_tags['country'] = self.content['country']
+        #freeform_tags['releasedate'] = self.content['releasedate']
+
+        return common_tags, freeform_tags
+
+    def folders(self):
+        artist = self.content['artist']
+        album = self.content['date'] + ' - ' + self.content['title'] + ' (' + self.content['label'] + ' - ' + self.content['catalog'] + ')'
+        return artist, album
+
+    def complete(self):
+        '''Check MetaFile for containing all data'''
+        self.__init__(self.path)
+        for l in self.rawcontent.splitlines():
+            if re.search('^[0-9A-Za-z]+?\t$', l):
+                return False
+        return True
+
+    def count_tracks(self):
+        '''Returns tracks count'''
+        return len(self.content['tracks']) - self.content['tracks'].count(None)
+
+
+class WaveFile(File):
+    def __init__(self, path, track_nr=None):
+        super(WaveFile, self).__init__(path)
+
+        self.track_nr = track_nr
+        self.type = WAVE_FILE_TYPE_BY_EXTENSION[self.name_ext]
+
+    def filename(self, ext=None, cmd=False):
+        title = CUE_META.content['tracks'][self.track_nr]['title']
+
+        if ext:
+            if ext[0] != '.':
+                ext = '.' + ext
+        else:
+            ext = self.name_ext
+
+        f_name = int_to_str(self.track_nr) + ' - ' + title + ext
+
+        if cmd:
+            f_name = f_name.replace(' ', '\ ')
+
+        f_name = f_name.replace('!', '')
+        f_name = f_name.replace('?', '')
+        f_name = f_name.replace('/', ';')
+
+        return f_name
+
+    def tag(self):
+        if self.type == 'Free Lossless Audio Codec':
+            f = FLAC(self.name)
+            tags = CUE_META.flac_tags(self.track_nr)
+            for t in tags[0]:
+                f[t] = tags[0][t]
+            f.save()
+
+    def mutagen(self):
+        if self.type == 'Free Lossless Audio Codec':
+            return FLAC(self.name)
+
+def split(albumpath):
+    global CUE_META
+    os.chdir(albumpath)
+    base_dir = Directory(os.getcwd())
+
+    cue = None
+    wave = None
+
+    # determining correct cue file
+    # if perfect match found
+    for _cue in base_dir.filter('CueFile'):
+        for _wave in base_dir.filter('WaveFile'):
+            if _cue.header['file'] == _wave.name:
+                logger.info('CUE Sheet found: %s', _cue.name)
+                logger.info('Music file found: %s', _wave.name)
+                cue = _cue
+                wave = _wave
+    # if no perfect match found then try without extensions
+    if not cue and not wave:
+        logger.info('No match for music files, trying to match without extensions...')
+        for _cue in base_dir.filter('CueFile'):
+            for _wave in base_dir.filter('WaveFile'):
+                if ''.join(os.path.splitext(_cue.header['file'])[:-1]) == _wave.name_name:
+                    logger.info('Possible CUE Sheet found: %s', _cue.name)
+                    logger.info('CUE Sheet refers music file: %s', _cue.header['file'])
+                    logger.info('Possible Music file found: %s', _wave.name)
+                    cue = _cue
+                    wave = _wave
+                    cue.header['file'] = wave.name
+    # if still no match then raise an exception
+    if not cue and not wave:
+        raise ValueError('No music file match found!')
+
+    # Split with xld or shntool
+    splitter = 'shntool'
+    xldprofile = None
+
+    # use xld profile to split cue
+    if headphones.CONFIG.ENCODER == 'xld' and headphones.CONFIG.MUSIC_ENCODER and headphones.CONFIG.XLDPROFILE:
+        import getXldProfile
+        xldprofile, xldformat, _ = getXldProfile.getXldProfile(headphones.CONFIG.XLDPROFILE)
+        if not xldformat:
+            raise ValueError('Details for xld profile "%s" not found, cannot split cue' % (xldprofile))
+        else:
+            if headphones.CONFIG.ENCODERFOLDER:
+                splitter = os.path.join(headphones.CONFIG.ENCODERFOLDER, 'xld')
+            else:
+                splitter = 'xld'
+    # use standard xld command to split cue
+    elif sys.platform == 'darwin':
+        splitter = 'xld'
+        if not check_splitter(splitter):
+            splitter = 'shntool'
+
+    if splitter == 'shntool' and not check_splitter(splitter):
+            raise ValueError('Command not found, ensure shntool or xld installed')
+
+    # Determine if file can be split
+    if wave.name_ext not in WAVE_FILE_TYPE_BY_EXTENSION.keys():
+        raise ValueError('Cannot split, audio file has unsupported extension')
+
+    # Split with xld
+    if 'xld' in splitter:
+        cmd = [splitter]
+        cmd.extend([wave.name])
+        cmd.extend(['-c'])
+        cmd.extend([cue.name])
+        if xldprofile:
+            cmd.extend(['--profile'])
+            cmd.extend([xldprofile])
+        else:
+            cmd.extend(['-f'])
+            cmd.extend(['flac'])
+        cmd.extend(['-o'])
+        cmd.extend([base_dir.path])
+        split = split_baby(wave.name, cmd)
+    else:
+
+        # Split with shntool
+
+        # generate temporary metafile describing the cue
+        with open(ALBUM_META_FILE_NAME, mode='w') as meta_file:
+            meta_file.write(cue.get_meta())
+        base_dir.content.append(MetaFile(os.path.abspath(ALBUM_META_FILE_NAME)))
+        # check metafile for completeness
+        if not base_dir.filter('MetaFile'):
+            raise ValueError('Cue Meta file {0} missing!'.format(ALBUM_META_FILE_NAME))
+        else:
+            CUE_META = base_dir.filter('MetaFile')[0]
+
+        with open(SPLIT_FILE_NAME, mode='w') as split_file:
+            split_file.write(cue.breakpoints())
+
+        if headphones.CONFIG.CUE_SPLIT_SHNTOOL_PATH:
+            cmd = [os.path.join(headphones.CONFIG.CUE_SPLIT_SHNTOOL_PATH, 'shntool')]
+        else:
+            cmd = ['shntool']
+
+        cmd.extend(['split'])
+        cmd.extend(['-f'])
+        cmd.extend([SPLIT_FILE_NAME])
+        cmd.extend(['-o'])
+        cmd.extend([wave.name_ext.lstrip('.')])
+        cmd.extend([wave.name])
+        split = split_baby(wave.name, cmd)
+        os.remove(SPLIT_FILE_NAME)
+        base_dir.update()
+
+        # tag FLAC files
+        if split and CUE_META.count_tracks() == len(base_dir.tracks(ext='.flac', split=True)):
+            for t in base_dir.tracks(ext='.flac', split=True):
+                logger.info('Tagging %s...', t.name)
+                t.tag()
+
+        # rename files
+        if split and CUE_META.count_tracks() == len(base_dir.tracks(ext=wave.name_ext, split=True)):
+            for t in base_dir.tracks(ext=wave.name_ext, split=True):
+                if t.name != t.filename():
+                    logger.info('Renaming %s to %s...', t.name, t.filename())
+                    os.rename(t.name, t.filename())
+
+        os.remove(ALBUM_META_FILE_NAME)
+
+    if not split:
+        raise ValueError('Failed to split, check logs')
+    else:
+        # Rename original file
+        os.rename(wave.name, wave.name + '.original')
+        return True
@@ -21,23 +21,24 @@ from __future__ import with_statement

 import os
 import sqlite3
-import threading
-import time

 import headphones

 from headphones import logger

+
 def dbFilename(filename="headphones.db"):

    return os.path.join(headphones.DATA_DIR, filename)

+
 def getCacheSize():
    #this will protect against typecasting problems produced by empty string and None settings
-    if not headphones.CACHE_SIZEMB:
+    if not headphones.CONFIG.CACHE_SIZEMB:
        #sqlite will work with this (very slowly)
        return 0
-    return int(headphones.CACHE_SIZEMB)
+    return int(headphones.CONFIG.CACHE_SIZEMB)
+

 class DBConnection:

@@ -48,25 +49,25 @@ class DBConnection:
        #don't wait for the disk to finish writing
        self.connection.execute("PRAGMA synchronous = OFF")
        #journal disabled since we never do rollbacks
-        self.connection.execute("PRAGMA journal_mode = %s" % headphones.JOURNAL_MODE)
+        self.connection.execute("PRAGMA journal_mode = %s" % headphones.CONFIG.JOURNAL_MODE)
        #64mb of cache memory,probably need to make it user configurable
-        self.connection.execute("PRAGMA cache_size=-%s" % (getCacheSize()*1024))
+        self.connection.execute("PRAGMA cache_size=-%s" % (getCacheSize() * 1024))
        self.connection.row_factory = sqlite3.Row

    def action(self, query, args=None):

-        if query == None:
+        if query is None:
            return

        sqlResult = None
-        
+
        try:
            with self.connection as c:
-                if args == None:
+                if args is None:
                    sqlResult = c.execute(query)
                else:
                    sqlResult = c.execute(query, args)
-                    
+
        except sqlite3.OperationalError, e:
            if "unable to open database file" in e.message or "database is locked" in e.message:
                logger.warn('Database Error: %s', e)
@@ -77,14 +78,14 @@ class DBConnection:
        except sqlite3.DatabaseError, e:
            logger.error('Fatal Error executing %s :: %s', query, e)
            raise
-        
+
        return sqlResult

    def select(self, query, args=None):

        sqlResults = self.action(query, args).fetchall()
-        
-        if sqlResults == None or sqlResults == [None]:
+
+        if sqlResults is None or sqlResults == [None]:
            return []

        return sqlResults
@@ -93,13 +94,18 @@ class DBConnection:

        changesBefore = self.connection.total_changes

-        genParams = lambda myDict : [x + " = ?" for x in myDict.keys()]
+        genParams = lambda myDict: [x + " = ?" for x in myDict.keys()]

-        query = "UPDATE "+tableName+" SET " + ", ".join(genParams(valueDict)) + " WHERE " + " AND ".join(genParams(keyDict))
+        update_query = "UPDATE " + tableName + " SET " + ", ".join(genParams(valueDict)) + " WHERE " + " AND ".join(genParams(keyDict))

-        self.action(query, valueDict.values() + keyDict.values())
+        self.action(update_query, valueDict.values() + keyDict.values())

        if self.connection.total_changes == changesBefore:
-            query = "INSERT INTO "+tableName+" (" + ", ".join(valueDict.keys() + keyDict.keys()) + ")" + \
-                        " VALUES (" + ", ".join(["?"] * len(valueDict.keys() + keyDict.keys())) + ")"
-            self.action(query, valueDict.values() + keyDict.values())
+            insert_query = (
+                "INSERT INTO " + tableName + " (" + ", ".join(valueDict.keys() + keyDict.keys()) + ")" +
+                " VALUES (" + ", ".join(["?"] * len(valueDict.keys() + keyDict.keys())) + ")"
+            )
+            try:
+                self.action(insert_query, valueDict.values() + keyDict.values())
+            except sqlite3.IntegrityError:
+                logger.info('Queries failed: %s and %s', update_query, insert_query)
@@ -13,11 +13,13 @@
 #  You should have received a copy of the GNU General Public License
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

+
 class HeadphonesException(Exception):
    """
    Generic Headphones Exception - should never be thrown, only subclassed
    """

+
 class NewzbinAPIThrottled(HeadphonesException):
    """
    Newzbin has throttled us, deal with it
@@ -1,32 +1,31 @@
 import os.path
-import plistlib
-import sys
-import xml.parsers.expat as expat
-import commands
+import biplist
 from headphones import logger

+
 def getXldProfile(xldProfile):
+
    xldProfileNotFound = xldProfile
-    expandedPath = os.path.expanduser('~/Library/Preferences/jp.tmkk.XLD.plist')
-    try:
-        preferences = plistlib.Plist.fromFile(expandedPath)
-    except (expat.ExpatError):
-        os.system("/usr/bin/plutil -convert xml1 %s" % expandedPath )
-        try:
-            preferences = plistlib.Plist.fromFile(expandedPath)
-        except (ImportError):
-            os.system("/usr/bin/plutil -convert binary1 %s" % expandedPath )
-            logger.info('The plist at "%s" has a date in it, and therefore is not useable.' % expandedPath)
-            return(xldProfileNotFound, None, None)
-    except (ImportError):
-        logger.info('The plist at "%s" has a date in it, and therefore is not useable.' % expandedPath)
-    except:
-        logger.info('Unexpected error:', sys.exc_info()[0])
+
+    expanded = os.path.expanduser('~/Library/Preferences/jp.tmkk.XLD.plist')
+    if not os.path.isfile(expanded):
+        logger.warn("Could not find xld preferences at: %s", expanded)
        return(xldProfileNotFound, None, None)

-    xldProfile = xldProfile.lower()
-    profiles = preferences.get('Profiles')
+    # Get xld preferences plist
+    try:
+        preferences = biplist.readPlist(expanded)
+    except (biplist.InvalidPlistException, biplist.NotBinaryPlistException), e:
+        logger.error("Error reading xld preferences plist: %s", e)
+        return(xldProfileNotFound, None, None)

+    if not isinstance(preferences, dict):
+        logger.error("Error reading xld preferences plist, not a dict: %r", preferences)
+        return(xldProfileNotFound, None, None)
+
+    profiles = preferences.get('Profiles', []) # pylint:disable=E1103
+
+    xldProfile = xldProfile.lower()
    for profile in profiles:

        profilename = profile.get('XLDProfileManager_ProfileName')
@@ -178,4 +177,4 @@ def getXldProfile(xldProfile):

            return(xldProfileForCmd, xldFormat, xldBitrate)

-    return(xldProfileNotFound, None, None)
+    return(xldProfileNotFound, None, None)
@@ -13,17 +13,19 @@
 #  You should have received a copy of the GNU General Public License
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

-import os
-import re
-import time
-import shutil
-import datetime
-import headphones
-import unicodedata
-import sys
+from beets.mediafile import MediaFile, FileTypeError, UnreadableFileError

 from operator import itemgetter
-from beets.mediafile import MediaFile, FileTypeError, UnreadableFileError
+
+import unicodedata
+import headphones
+import datetime
+import fnmatch
+import shutil
+import time
+import sys
+import re
+import os

 # Modified from https://github.com/Verrus/beets-plugin-featInTitle
 RE_FEATURING = re.compile(r"[fF]t\.|[fF]eaturing|[fF]eat\.|\b[wW]ith\b|&|vs\.")
@@ -31,8 +33,9 @@ RE_FEATURING = re.compile(r"[fF]t\.|[fF]eaturing|[fF]eat\.|\b[wW]ith\b|&|vs\.")
 RE_CD_ALBUM = re.compile(r"\(?((CD|disc)\s*[0-9]+)\)?", re.I)
 RE_CD = re.compile(r"^(CD|dics)\s*[0-9]+$", re.I)

+
 def multikeysort(items, columns):
-    comparers = [ ((itemgetter(col[1:].strip()), -1) if col.startswith('-') else (itemgetter(col.strip()), 1)) for col in columns]
+    comparers = [((itemgetter(col[1:].strip()), -1) if col.startswith('-') else (itemgetter(col.strip()), 1)) for col in columns]

    def comparer(left, right):
        for fn, mult in comparers:
@@ -44,12 +47,14 @@ def multikeysort(items, columns):

    return sorted(items, cmp=comparer)

+
 def checked(variable):
    if variable:
        return 'Checked'
    else:
        return ''

+
 def radio(variable, pos):

    if variable == pos:
@@ -57,40 +62,42 @@ def radio(variable, pos):
    else:
        return ''

+
 def latinToAscii(unicrap):
    """
    From couch potato
    """
-    xlate = {0xc0:'A', 0xc1:'A', 0xc2:'A', 0xc3:'A', 0xc4:'A', 0xc5:'A',
-        0xc6:'Ae', 0xc7:'C',
-        0xc8:'E', 0xc9:'E', 0xca:'E', 0xcb:'E', 0x86:'e',
-        0xcc:'I', 0xcd:'I', 0xce:'I', 0xcf:'I',
-        0xd0:'Th', 0xd1:'N',
-        0xd2:'O', 0xd3:'O', 0xd4:'O', 0xd5:'O', 0xd6:'O', 0xd8:'O',
-        0xd9:'U', 0xda:'U', 0xdb:'U', 0xdc:'U',
-        0xdd:'Y', 0xde:'th', 0xdf:'ss',
-        0xe0:'a', 0xe1:'a', 0xe2:'a', 0xe3:'a', 0xe4:'a', 0xe5:'a',
-        0xe6:'ae', 0xe7:'c',
-        0xe8:'e', 0xe9:'e', 0xea:'e', 0xeb:'e', 0x0259:'e',
-        0xec:'i', 0xed:'i', 0xee:'i', 0xef:'i',
-        0xf0:'th', 0xf1:'n',
-        0xf2:'o', 0xf3:'o', 0xf4:'o', 0xf5:'o', 0xf6:'o', 0xf8:'o',
-        0xf9:'u', 0xfa:'u', 0xfb:'u', 0xfc:'u',
-        0xfd:'y', 0xfe:'th', 0xff:'y',
-        0xa1:'!', 0xa2:'{cent}', 0xa3:'{pound}', 0xa4:'{currency}',
-        0xa5:'{yen}', 0xa6:'|', 0xa7:'{section}', 0xa8:'{umlaut}',
-        0xa9:'{C}', 0xaa:'{^a}', 0xab:'<<', 0xac:'{not}',
-        0xad:'-', 0xae:'{R}', 0xaf:'_', 0xb0:'{degrees}',
-        0xb1:'{+/-}', 0xb2:'{^2}', 0xb3:'{^3}', 0xb4:"'",
-        0xb5:'{micro}', 0xb6:'{paragraph}', 0xb7:'*', 0xb8:'{cedilla}',
-        0xb9:'{^1}', 0xba:'{^o}', 0xbb:'>>',
-        0xbc:'{1/4}', 0xbd:'{1/2}', 0xbe:'{3/4}', 0xbf:'?',
-        0xd7:'*', 0xf7:'/'
-        }
+    xlate = {
+        0xc0: 'A', 0xc1: 'A', 0xc2: 'A', 0xc3: 'A', 0xc4: 'A', 0xc5: 'A',
+        0xc6: 'Ae', 0xc7: 'C',
+        0xc8: 'E', 0xc9: 'E', 0xca: 'E', 0xcb: 'E', 0x86: 'e',
+        0xcc: 'I', 0xcd: 'I', 0xce: 'I', 0xcf: 'I',
+        0xd0: 'Th', 0xd1: 'N',
+        0xd2: 'O', 0xd3: 'O', 0xd4: 'O', 0xd5: 'O', 0xd6: 'O', 0xd8: 'O',
+        0xd9: 'U', 0xda: 'U', 0xdb: 'U', 0xdc: 'U',
+        0xdd: 'Y', 0xde: 'th', 0xdf: 'ss',
+        0xe0: 'a', 0xe1: 'a', 0xe2: 'a', 0xe3: 'a', 0xe4: 'a', 0xe5: 'a',
+        0xe6: 'ae', 0xe7: 'c',
+        0xe8: 'e', 0xe9: 'e', 0xea: 'e', 0xeb: 'e', 0x0259: 'e',
+        0xec: 'i', 0xed: 'i', 0xee: 'i', 0xef: 'i',
+        0xf0: 'th', 0xf1: 'n',
+        0xf2: 'o', 0xf3: 'o', 0xf4: 'o', 0xf5: 'o', 0xf6: 'o', 0xf8: 'o',
+        0xf9: 'u', 0xfa: 'u', 0xfb: 'u', 0xfc: 'u',
+        0xfd: 'y', 0xfe: 'th', 0xff: 'y',
+        0xa1: '!', 0xa2: '{cent}', 0xa3: '{pound}', 0xa4: '{currency}',
+        0xa5: '{yen}', 0xa6: '|', 0xa7: '{section}', 0xa8: '{umlaut}',
+        0xa9: '{C}', 0xaa: '{^a}', 0xab: '<<', 0xac: '{not}',
+        0xad: '-', 0xae: '{R}', 0xaf: '_', 0xb0: '{degrees}',
+        0xb1: '{+/-}', 0xb2: '{^2}', 0xb3: '{^3}', 0xb4: "'",
+        0xb5: '{micro}', 0xb6: '{paragraph}', 0xb7: '*', 0xb8: '{cedilla}',
+        0xb9: '{^1}', 0xba: '{^o}', 0xbb: '>>',
+        0xbc: '{1/4}', 0xbd: '{1/2}', 0xbe: '{3/4}', 0xbf: '?',
+        0xd7: '*', 0xf7: '/'
+    }

    r = ''
    for i in unicrap:
-        if xlate.has_key(ord(i)):
+        if ord(i) in xlate:
            r += xlate[ord(i)]
        elif ord(i) >= 0x80:
            pass
@@ -98,9 +105,10 @@ def latinToAscii(unicrap):
            r += str(i)
    return r

+
 def convert_milliseconds(ms):

-    seconds = ms/1000
+    seconds = ms / 1000
    gmtime = time.gmtime(seconds)
    if seconds > 3600:
        minutes = time.strftime("%H:%M:%S", gmtime)
@@ -109,6 +117,7 @@ def convert_milliseconds(ms):

    return minutes

+
 def convert_seconds(s):

    gmtime = time.gmtime(s)
@@ -119,15 +128,18 @@ def convert_seconds(s):

    return minutes

+
 def today():
    today = datetime.date.today()
    yyyymmdd = datetime.date.isoformat(today)
    return yyyymmdd

+
 def now():
    now = datetime.datetime.now()
    return now.strftime("%Y-%m-%d %H:%M:%S")

+
 def get_age(date):

    try:
@@ -136,33 +148,43 @@ def get_age(date):
        return False

    try:
-        days_old = int(split_date[0])*365 + int(split_date[1])*30 + int(split_date[2])
+        days_old = int(split_date[0]) * 365 + int(split_date[1]) * 30 + int(split_date[2])
    except IndexError:
        days_old = False

    return days_old

+
 def bytes_to_mb(bytes):

-    mb = int(bytes)/1048576
+    mb = int(bytes) / 1048576
    size = '%.1f MB' % mb
    return size

+
 def mb_to_bytes(mb_str):
    result = re.search('^(\d+(?:\.\d+)?)\s?(?:mb)?', mb_str, flags=re.I)
    if result:
-        return int(float(result.group(1))*1048576)
+        return int(float(result.group(1)) * 1048576)
+

 def piratesize(size):
    split = size.split(" ")
    factor = float(split[0])
-    unit = split[1]
-    if unit == 'MiB':
+    unit = split[1].upper()
+    
+    if unit == 'MIB':
        size = factor * 1048576
-    elif unit == 'GiB':
+    elif unit == 'MB':
+        size = factor * 1000000
+    elif unit == 'GIB':
        size = factor * 1073741824
-    elif unit == 'KiB':
+    elif unit == 'GB':
+        size = factor * 1000000000
+    elif unit == 'KIB':
        size = factor * 1024
+    elif unit == 'KB':
+        size = factor * 1000
    elif unit == "B":
        size = factor
    else:
@@ -170,6 +192,7 @@ def piratesize(size):

    return size

+
 def replace_all(text, dic, normalize=False):

    if not text:
@@ -177,13 +200,17 @@ def replace_all(text, dic, normalize=False):

    for i, j in dic.iteritems():
        if normalize:
-            if sys.platform == 'darwin':
-                j = unicodedata.normalize('NFD', j)
-            else:
-                j = unicodedata.normalize('NFC', j)
+            try:
+                if sys.platform == 'darwin':
+                    j = unicodedata.normalize('NFD', j)
+                else:
+                    j = unicodedata.normalize('NFC', j)
+            except TypeError:
+                j = unicodedata.normalize('NFC', j.decode(headphones.SYS_ENCODING, 'replace'))
        text = text.replace(i, j)
    return text

+
 def replace_illegal_chars(string, type="file"):
    if type == "file":
        string = re.sub('[\?"*:|<>/]', '_', string)
@@ -192,6 +219,7 @@ def replace_illegal_chars(string, type="file"):

    return string

+
 def cleanName(string):

    pass1 = latinToAscii(string).lower()
@@ -199,6 +227,7 @@ def cleanName(string):

    return out_string

+
 def cleanTitle(title):

    title = re.sub('[\.\-\/\_]', ' ', title).lower()
@@ -210,6 +239,7 @@ def cleanTitle(title):

    return title

+
 def split_path(f):
    """
    Split a path into components, starting with the drive letter (if any). Given
@@ -219,7 +249,7 @@ def split_path(f):
    components = []
    drive, path = os.path.splitdrive(f)

-    # Stip the folder from the path, iterate until nothing is left
+    # Strip the folder from the path, iterate until nothing is left
    while True:
        path, folder = os.path.split(path)

@@ -241,6 +271,7 @@ def split_path(f):
    # Done
    return components

+
 def expand_subfolders(f):
    """
    Try to expand a given folder and search for subfolders containing media
@@ -269,7 +300,7 @@ def expand_subfolders(f):
        return

    # Split into path components
-    media_folders = [ split_path(media_folder) for media_folder in media_folders ]
+    media_folders = [split_path(media_folder) for media_folder in media_folders]

    # Correct folder endings such as CD1 etc.
    for index, media_folder in enumerate(media_folders):
@@ -277,7 +308,7 @@ def expand_subfolders(f):
            media_folders[index] = media_folders[index][:-1]

    # Verify the result by computing path depth relative to root.
-    path_depths = [ len(media_folder) for media_folder in media_folders ]
+    path_depths = [len(media_folder) for media_folder in media_folders]
    difference = max(path_depths) - min(path_depths)

    if difference > 0:
@@ -287,15 +318,15 @@ def expand_subfolders(f):
        # directory may contain separate CD's and maybe some extra's. The
        # structure may look like X albums at same depth, and (one or more)
        # extra folders with a higher depth.
-        extra_media_folders = [ media_folder[:min(path_depths)] for media_folder in media_folders if len(media_folder) > min(path_depths) ]
-        extra_media_folders = list(set([ os.path.join(*media_folder) for media_folder in extra_media_folders ]))
+        extra_media_folders = [media_folder[:min(path_depths)] for media_folder in media_folders if len(media_folder) > min(path_depths)]
+        extra_media_folders = list(set([os.path.join(*media_folder) for media_folder in extra_media_folders]))

        logger.info("Please look at the following folder(s), since they cause the depth difference: %s", extra_media_folders)
        return

    # Convert back to paths and remove duplicates, which may be there after
    # correcting the paths
-    media_folders = list(set([ os.path.join(*media_folder) for media_folder in media_folders ]))
+    media_folders = list(set([os.path.join(*media_folder) for media_folder in media_folders]))

    # Don't return a result if the number of subfolders is one. In this case,
    # this algorithm will not improve processing and will likely interfere
@@ -307,23 +338,53 @@ def expand_subfolders(f):
    logger.debug("Expanded subfolders in folder: %s", media_folders)
    return media_folders

+
+def path_match_patterns(path, patterns):
+    """
+    Check if a path matches one or more patterns. The whole path will be
+    matched be matched against the patterns.
+    """
+
+    for pattern in patterns:
+        if fnmatch.fnmatch(path, pattern):
+            return True
+
+    # No match
+    return False
+
+
+def path_filter_patterns(paths, patterns, root=None):
+    """
+    Scan for ignored paths based on glob patterns. Note that the whole path
+    will be matched, therefore paths should only contain the relative paths.
+
+    The root is optional, and only used for producing meaningful debug info.
+    """
+
+    from headphones import logger
+
+    ignored = 0
+
+    for path in paths[:]:
+        if path_match_patterns(path, patterns):
+            logger.debug("Path ignored by pattern: %s",
+                os.path.join(root or "", path))
+
+            ignored += 1
+            paths.remove(path)
+
+    # Return number of ignored paths
+    return ignored
+
+
 def extract_data(s):

    s = s.replace('_', ' ')

    #headphones default format
-    pattern = re.compile(r'(?P<name>.*?)\s\-\s(?P<album>.*?)\s\[(?P<year>.*?)\]', re.VERBOSE)
+    pattern = re.compile(r'(?P<name>.*?)\s\-\s(?P<album>.*?)\s[\[\(](?P<year>.*?)[\]\)]', re.VERBOSE)
    match = pattern.match(s)

-    if match:
-        name = match.group("name")
-        album = match.group("album")
-        year = match.group("year")
-        return (name, album, year)
-
-    #newzbin default format
-    pattern = re.compile(r'(?P<name>.*?)\s\-\s(?P<album>.*?)\s\((?P<year>\d+?\))', re.VERBOSE)
-    match = pattern.match(s)
    if match:
        name = match.group("name")
        album = match.group("album")
@@ -343,6 +404,7 @@ def extract_data(s):
    else:
        return (None, None, None)

+
 def extract_metadata(f):
    """
    Scan all files in the given directory and decide on an artist, album and
@@ -392,9 +454,9 @@ def extract_metadata(f):
        return (None, None, None)

    # Count distinct values
-    artists = list(set([ x[0] for x in results ]))
-    albums = list(set([ x[1] for x in results ]))
-    years = list(set([ x[2] for x in results ]))
+    artists = list(set([x[0] for x in results]))
+    albums = list(set([x[1] for x in results]))
+    years = list(set([x[2] for x in results]))

    # Remove things such as CD2 from album names
    if len(albums) > 1:
@@ -422,8 +484,8 @@ def extract_metadata(f):

    # (Lots of) different artists. Could be a featuring album, so test for this.
    if len(artists) > 1 and len(albums) == 1:
-        split_artists = [ RE_FEATURING.split(artist) for artist in artists ]
-        featurings = [ len(split_artist) - 1 for split_artist in split_artists ]
+        split_artists = [RE_FEATURING.split(x) for x in artists]
+        featurings = [len(split_artist) - 1 for split_artist in split_artists]
        logger.info("Album seem to feature %d different artists", sum(featurings))

        if sum(featurings) > 0:
@@ -441,6 +503,79 @@ def extract_metadata(f):

    return (None, None, None)

+
+def get_downloaded_track_list(albumpath):
+    """
+     Return a list of audio files for the given directory.
+     """
+    downloaded_track_list = []
+
+    for root, dirs, files in os.walk(albumpath):
+        for _file in files:
+            extension = os.path.splitext(_file)[1].lower()[1:]
+            if extension in headphones.MEDIA_FORMATS:
+                downloaded_track_list.append(os.path.join(root, _file))
+
+    return downloaded_track_list
+
+
+def preserve_torrent_directory(albumpath):
+    """
+    Copy torrent directory to headphones-modified to keep files for seeding.
+    """
+    from headphones import logger
+    new_folder = os.path.join(albumpath, 'headphones-modified'.encode(headphones.SYS_ENCODING, 'replace'))
+    logger.info("Copying files to 'headphones-modified' subfolder to preserve downloaded files for seeding")
+    try:
+        shutil.copytree(albumpath, new_folder)
+        return new_folder
+    except Exception as e:
+        logger.warn("Cannot copy/move files to temp folder: " + \
+                    new_folder.decode(headphones.SYS_ENCODING, 'replace') + \
+                    ". Not continuing. Error: " + str(e))
+        return None
+
+
+def cue_split(albumpath):
+    """
+     Attempts to check and split audio files by a cue for the given directory.
+     """
+    # Walk directory and scan all media files
+    count = 0
+    cue_count = 0
+    cue_dirs = []
+
+    for root, dirs, files in os.walk(albumpath):
+        for _file in files:
+            extension = os.path.splitext(_file)[1].lower()[1:]
+            if extension in headphones.MEDIA_FORMATS:
+                count += 1
+            elif extension == 'cue':
+                cue_count += 1
+                if root not in cue_dirs:
+                    cue_dirs.append(root)
+
+    # Split cue
+    if cue_count and cue_count >= count and cue_dirs:
+
+        from headphones import logger, cuesplit
+        logger.info("Attempting to split audio files by cue")
+
+        cwd = os.getcwd()
+        for cue_dir in cue_dirs:
+            try:
+                cuesplit.split(cue_dir)
+            except Exception as e:
+                os.chdir(cwd)
+                logger.warn("Cue not split: " + str(e))
+                return False
+
+        os.chdir(cwd)
+        return True
+
+    return False
+
+
 def extract_logline(s):
    # Default log format
    pattern = re.compile(r'(?P<timestamp>.*?)\s\-\s(?P<level>.*?)\s*\:\:\s(?P<thread>.*?)\s\:\s(?P<message>.*)', re.VERBOSE)
@@ -454,14 +589,11 @@ def extract_logline(s):
    else:
        return None

+
 def extract_song_data(s):
+    from headphones import logger

    #headphones default format
-    music_dir = headphones.MUSIC_DIR
-    folder_format = headphones.FOLDER_FORMAT
-    file_format = headphones.FILE_FORMAT
-
-    full_format = os.path.join(headphones.MUSIC_DIR)
    pattern = re.compile(r'(?P<name>.*?)\s\-\s(?P<album>.*?)\s\[(?P<year>.*?)\]', re.VERBOSE)
    match = pattern.match(s)

@@ -485,6 +617,7 @@ def extract_song_data(s):
        logger.info("Couldn't parse %s into a valid Newbin format", s)
        return (name, album, year)

+
 def smartMove(src, dest, delete=True):

    from headphones import logger
@@ -506,7 +639,7 @@ def smartMove(src, dest, delete=True):
                try:
                    os.rename(src, os.path.join(source_dir, newfile))
                    filename = newfile
-                except Exception, e:
+                except Exception as e:
                    logger.warn('Error renaming %s: %s', src.decode(headphones.SYS_ENCODING, 'replace'), e)
                break

@@ -516,41 +649,82 @@ def smartMove(src, dest, delete=True):
        else:
            shutil.copy(os.path.join(source_dir, filename), os.path.join(dest, filename))
            return True
-    except Exception, e:
+    except Exception as e:
        logger.warn('Error moving file %s: %s', filename.decode(headphones.SYS_ENCODING, 'replace'), e)

+def walk_directory(basedir, followlinks=True):
+    """
+    Enhanced version of 'os.walk' where symlink directores are traversed, but
+    with care. In case a folder is already processed, don't traverse it again.
+    """
+
+    import logger
+
+    # Add the base path, because symlinks poiting to the basedir should not be
+    # traversed again.
+    traversed = [os.path.abspath(basedir)]
+
+    def _inner(root, directories, files):
+        for directory in directories:
+            path = os.path.join(root, directory)
+
+            if followlinks and os.path.islink(path):
+                real_path = os.path.abspath(os.readlink(path))
+
+                if real_path in traversed:
+                    logger.debug("Skipping '%s' since it is a symlink to "\
+                        "'%s', which is already visited.", path, real_path)
+                else:
+                    traversed.append(real_path)
+
+                    for args in os.walk(real_path):
+                        for result in _inner(*args):
+                            yield result
+
+        # Pass on actual result
+        yield root, directories, files
+
+    # Start traversing
+    for args in os.walk(basedir):
+        for result in _inner(*args):
+            yield result
+
 #########################
 #Sab renaming functions #
 #########################

 # TODO: Grab config values from sab to know when these options are checked. For now we'll just iterate through all combinations

+
 def sab_replace_dots(name):
-    return name.replace('.',' ')
+    return name.replace('.', ' ')
+
+
 def sab_replace_spaces(name):
-    return name.replace(' ','_')
+    return name.replace(' ', '_')
+

 def sab_sanitize_foldername(name):
    """ Return foldername with dodgy chars converted to safe ones
        Remove any leading and trailing dot and space characters
    """
    CH_ILLEGAL = r'\/<>?*|"'
-    CH_LEGAL   = r'++{}!@#`'
+    CH_LEGAL = r'++{}!@#`'

    FL_ILLEGAL = CH_ILLEGAL + ':\x92"'
-    FL_LEGAL   = CH_LEGAL +   "-''"
+    FL_LEGAL = CH_LEGAL + "-''"

    uFL_ILLEGAL = FL_ILLEGAL.decode('latin-1')
-    uFL_LEGAL   = FL_LEGAL.decode('latin-1')
+    uFL_LEGAL = FL_LEGAL.decode('latin-1')

    if not name:
        return name
    if isinstance(name, unicode):
        illegal = uFL_ILLEGAL
-        legal   = uFL_LEGAL
+        legal = uFL_LEGAL
    else:
        illegal = FL_ILLEGAL
-        legal   = FL_LEGAL
+        legal = FL_LEGAL

    lst = []
    for ch in name.strip():
@@ -579,34 +753,33 @@ def split_string(mystring, splitvar=','):

 def create_https_certificates(ssl_cert, ssl_key):
    """
-    Stolen from SickBeard (http://github.com/midgetspy/Sick-Beard):
-    Create self-signed HTTPS certificares and store in paths 'ssl_cert' and 'ssl_key'
+    Create a pair of self-signed HTTPS certificares and store in them in
+    'ssl_cert' and 'ssl_key'. Method assumes pyOpenSSL is installed.
+
+    This code is stolen from SickBeard (http://github.com/midgetspy/Sick-Beard).
    """
+
    from headphones import logger

-    try:
-        from OpenSSL import crypto
-        from lib.certgen import createKeyPair, createCertRequest, createCertificate, TYPE_RSA, serial
-    except:
-        logger.warn("pyOpenSSL module missing, please install to enable HTTPS")
-        return False
+    from OpenSSL import crypto
+    from certgen import createKeyPair, createCertRequest, createCertificate, \
+        TYPE_RSA, serial

    # Create the CA Certificate
-    cakey = createKeyPair(TYPE_RSA, 1024)
-    careq = createCertRequest(cakey, CN='Certificate Authority')
-    cacert = createCertificate(careq, (careq, cakey), serial, (0, 60*60*24*365*10)) # ten years
+    cakey = createKeyPair(TYPE_RSA, 2048)
+    careq = createCertRequest(cakey, CN="Certificate Authority")
+    cacert = createCertificate(careq, (careq, cakey), serial, (0, 60 * 60 * 24 * 365 * 10)) # ten years

-    cname = 'Headphones'
-    pkey = createKeyPair(TYPE_RSA, 1024)
-    req = createCertRequest(pkey, CN=cname)
-    cert = createCertificate(req, (cacert, cakey), serial, (0, 60*60*24*365*10)) # ten years
+    pkey = createKeyPair(TYPE_RSA, 2048)
+    req = createCertRequest(pkey, CN="Headphones")
+    cert = createCertificate(req, (cacert, cakey), serial, (0, 60 * 60 * 24 * 365 * 10)) # ten years

    # Save the key and certificate to disk
    try:
-        with open(ssl_key, 'w') as f:
-            f.write(crypto.dump_privatekey(crypto.FILETYPE_PEM, pkey))
-        with open(ssl_cert, 'w') as f:
-            f.write(crypto.dump_certificate(crypto.FILETYPE_PEM, cert))
+        with open(ssl_key, "w") as fp:
+            fp.write(crypto.dump_privatekey(crypto.FILETYPE_PEM, pkey))
+        with open(ssl_cert, "w") as fp:
+            fp.write(crypto.dump_certificate(crypto.FILETYPE_PEM, cert))
    except IOError as e:
        logger.error("Error creating SSL key and certificate: %s", e)
        return False
@@ -13,17 +13,15 @@
 #  You should have received a copy of the GNU General Public License
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

-from headphones import logger, helpers, db, mb, lastfm
+from headphones import logger, helpers, db, mb, lastfm, metacritic

 from beets.mediafile import MediaFile

-import os
 import time
-import threading
 import headphones

 blacklisted_special_artist_names = ['[anonymous]', '[data]', '[no artist]',
-    '[traditional]','[unknown]','Various Artists']
+    '[traditional]', '[unknown]', 'Various Artists']
 blacklisted_special_artists = ['f731ccc4-e22a-43af-a747-64213329e088',
    '33cf029c-63b0-41a0-9855-be2a3665fb3b',
    '314e1c25-dde7-4e4d-b2f4-0a7b9f7c56dc',
@@ -32,6 +30,7 @@ blacklisted_special_artists = ['f731ccc4-e22a-43af-a747-64213329e088',
    '125ec42a-7229-4250-afc5-e057484327fe',
    '89ad4ac3-39f7-470e-963a-56509c546377']

+
 def is_exists(artistid):
    myDB = db.DBConnection()

@@ -49,10 +48,9 @@ def artistlist_to_mbids(artistlist, forced=False):

    for artist in artistlist:

-        if not artist and not (artist == ' '):
+        if not artist and artist != ' ':
            continue

-
        # If adding artists through Manage New Artists, they're coming through as non-unicode (utf-8?)
        # and screwing everything up
        if not isinstance(artist, unicode):
@@ -105,13 +103,15 @@ def artistlist_to_mbids(artistlist, forced=False):
    except Exception as e:
        logger.warn('Failed to update arist information from Last.fm: %s' % e)

+
 def addArtistIDListToDB(artistidlist):
    # Used to add a list of artist IDs to the database in a single thread
    logger.debug("Importer: Adding artist ids %s" % artistidlist)
    for artistid in artistidlist:
        addArtisttoDB(artistid)

-def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
+
+def addArtisttoDB(artistid, extrasonly=False, forcefull=False, type="artist"):

    # Putting this here to get around the circular import. We're using this to update thumbnails for artist/albums
    from headphones import cache
@@ -131,23 +131,30 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):

    # We need the current minimal info in the database instantly
    # so we don't throw a 500 error when we redirect to the artistPage
-    controlValueDict = {"ArtistID":     artistid}
+    controlValueDict = {"ArtistID": artistid}

    # Don't replace a known artist name with an "Artist ID" placeholder
    dbartist = myDB.action('SELECT * FROM artists WHERE ArtistID=?', [artistid]).fetchone()

    # Only modify the Include Extras stuff if it's a new artist. We need it early so we know what to fetch
    if not dbartist:
-        newValueDict = {"ArtistName":   "Artist ID: %s" % (artistid),
-                        "Status":       "Loading",
-                        "IncludeExtras": headphones.INCLUDE_EXTRAS,
-                        "Extras":        headphones.EXTRAS }
+        newValueDict = {"ArtistName": "Artist ID: %s" % (artistid),
+                        "Status": "Loading",
+                        "IncludeExtras": headphones.CONFIG.INCLUDE_EXTRAS,
+                        "Extras": headphones.CONFIG.EXTRAS}
+        if type=="series":
+            newValueDict['Type'] = "series"
    else:
-        newValueDict = {"Status":   "Loading"}
+        newValueDict = {"Status": "Loading"}
+        if dbartist["Type"] == "series":
+            type = "series"

    myDB.upsert("artists", newValueDict, controlValueDict)

-    artist = mb.getArtist(artistid, extrasonly)
+    if type=="series":
+        artist = mb.getSeries(artistid)
+    else:
+        artist = mb.getArtist(artistid, extrasonly)

    if artist and artist.get('artist_name') in blacklisted_special_artist_names:
        logger.warn('Cannot import blocked special purpose artist: %s' % artist.get('artist_name'))
@@ -160,10 +167,10 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
    if not artist:
        logger.warn("Error fetching artist info. ID: " + artistid)
        if dbartist is None:
-            newValueDict = {"ArtistName":   "Fetch failed, try refreshing. (%s)" % (artistid),
-                    "Status":   "Active"}
+            newValueDict = {"ArtistName": "Fetch failed, try refreshing. (%s)" % (artistid),
+                    "Status": "Active"}
        else:
-            newValueDict = {"Status":   "Active"}
+            newValueDict = {"Status": "Active"}
        myDB.upsert("artists", newValueDict, controlValueDict)
        return

@@ -172,13 +179,12 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
    else:
        sortname = artist['artist_name']

-
    logger.info(u"Now adding/updating: " + artist['artist_name'])
-    controlValueDict = {"ArtistID":     artistid}
-    newValueDict = {"ArtistName":       artist['artist_name'],
-                    "ArtistSortName":   sortname,
-                    "DateAdded":        helpers.today(),
-                    "Status":           "Loading"}
+    controlValueDict = {"ArtistID": artistid}
+    newValueDict = {"ArtistName": artist['artist_name'],
+                    "ArtistSortName": sortname,
+                    "DateAdded": helpers.today(),
+                    "Status": "Loading"}

    myDB.upsert("artists", newValueDict, controlValueDict)

@@ -227,7 +233,7 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
        rgid = rg['id']
        skip_log = 0
        #Make a user configurable variable to skip update of albums with release dates older than this date (in days)
-        pause_delta = headphones.MB_IGNORE_AGE
+        pause_delta = headphones.CONFIG.MB_IGNORE_AGE

        rg_exists = myDB.action("SELECT * from albums WHERE AlbumID=?", [rg['id']]).fetchone()

@@ -240,27 +246,26 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
                check_release_date = None
                new_release_group = True

-
            if new_release_group:
                logger.info("[%s] Now adding: %s (New Release Group)" % (artist['artist_name'], rg['title']))
-                new_releases = mb.get_new_releases(rgid,includeExtras)
+                new_releases = mb.get_new_releases(rgid, includeExtras)

            else:
                if check_release_date is None or check_release_date == u"None":
                    logger.info("[%s] Now updating: %s (No Release Date)" % (artist['artist_name'], rg['title']))
-                    new_releases = mb.get_new_releases(rgid,includeExtras,True)
+                    new_releases = mb.get_new_releases(rgid, includeExtras, True)
                else:
                    if len(check_release_date) == 10:
                        release_date = check_release_date
                    elif len(check_release_date) == 7:
-                        release_date = check_release_date+"-31"
+                        release_date = check_release_date + "-31"
                    elif len(check_release_date) == 4:
-                        release_date = check_release_date+"-12-31"
+                        release_date = check_release_date + "-12-31"
                    else:
                        release_date = today
                    if helpers.get_age(today) - helpers.get_age(release_date) < pause_delta:
                        logger.info("[%s] Now updating: %s (Release Date <%s Days)", artist['artist_name'], rg['title'], pause_delta)
-                        new_releases = mb.get_new_releases(rgid,includeExtras,True)
+                        new_releases = mb.get_new_releases(rgid, includeExtras, True)
                    else:
                        logger.info("[%s] Skipping: %s (Release Date >%s Days)", artist['artist_name'], rg['title'], pause_delta)
                        skip_log = 1
@@ -273,7 +278,7 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
                new_releases = new_releases
        else:
            logger.info("[%s] Now adding/updating: %s (Comprehensive Force)", artist['artist_name'], rg['title'])
-            new_releases = mb.get_new_releases(rgid,includeExtras,forcefull)
+            new_releases = mb.get_new_releases(rgid, includeExtras, forcefull)

        if new_releases != 0:
            # Dump existing hybrid release since we're repackaging/replacing it
@@ -292,26 +297,26 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
            for items in find_hybrid_releases:
                if items['ReleaseID'] != rg['id']: #don't include hybrid information, since that's what we're replacing
                    hybrid_release_id = items['ReleaseID']
-                    newValueDict = {"ArtistID":         items['ArtistID'],
-                        "ArtistName":       items['ArtistName'],
-                        "AlbumTitle":       items['AlbumTitle'],
-                        "AlbumID":          items['AlbumID'],
-                        "AlbumASIN":        items['AlbumASIN'],
-                        "ReleaseDate":      items['ReleaseDate'],
-                        "Type":             items['Type'],
-                        "ReleaseCountry":   items['ReleaseCountry'],
-                        "ReleaseFormat":    items['ReleaseFormat']
+                    newValueDict = {"ArtistID": items['ArtistID'],
+                        "ArtistName": items['ArtistName'],
+                        "AlbumTitle": items['AlbumTitle'],
+                        "AlbumID": items['AlbumID'],
+                        "AlbumASIN": items['AlbumASIN'],
+                        "ReleaseDate": items['ReleaseDate'],
+                        "Type": items['Type'],
+                        "ReleaseCountry": items['ReleaseCountry'],
+                        "ReleaseFormat": items['ReleaseFormat']
                    }
                    find_hybrid_tracks = myDB.action("SELECT * from alltracks WHERE ReleaseID=?", [hybrid_release_id])
                    totalTracks = 1
                    hybrid_track_array = []
                    for hybrid_tracks in find_hybrid_tracks:
                        hybrid_track_array.append({
-                            'number':        hybrid_tracks['TrackNumber'],
-                            'title':         hybrid_tracks['TrackTitle'],
-                            'id':            hybrid_tracks['TrackID'],
+                            'number': hybrid_tracks['TrackNumber'],
+                            'title': hybrid_tracks['TrackTitle'],
+                            'id': hybrid_tracks['TrackID'],
                            #'url':           hybrid_tracks['TrackURL'],
-                            'duration':      hybrid_tracks['TrackDuration']
+                            'duration': hybrid_tracks['TrackDuration']
                            })
                        totalTracks += 1
                    newValueDict['ReleaseID'] = hybrid_release_id
@@ -325,21 +330,21 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
                logger.info('[%s] Packaging %s releases into hybrid title' % (artist['artist_name'], rg['title']))
            except Exception as e:
                errors = True
-                logger.warn('[%s] Unable to get hybrid release information for %s: %s' % (artist['artist_name'],rg['title'],e))
+                logger.warn('[%s] Unable to get hybrid release information for %s: %s' % (artist['artist_name'], rg['title'], e))
                continue

            # Use the ReleaseGroupID as the ReleaseID for the hybrid release to differentiate it
            # We can then use the condition WHERE ReleaseID == ReleaseGroupID to select it
            # The hybrid won't have a country or a format
-            controlValueDict = {"ReleaseID":  rg['id']}
+            controlValueDict = {"ReleaseID": rg['id']}

-            newValueDict = {"ArtistID":         artistid,
-                            "ArtistName":       artist['artist_name'],
-                            "AlbumTitle":       rg['title'],
-                            "AlbumID":          rg['id'],
-                            "AlbumASIN":        hybridrelease['AlbumASIN'],
-                            "ReleaseDate":      hybridrelease['ReleaseDate'],
-                            "Type":             rg['type']
+            newValueDict = {"ArtistID": artistid,
+                            "ArtistName": artist['artist_name'],
+                            "AlbumTitle": rg['title'],
+                            "AlbumID": rg['id'],
+                            "AlbumASIN": hybridrelease['AlbumASIN'],
+                            "ReleaseDate": hybridrelease['ReleaseDate'],
+                            "Type": rg['type']
                        }

            myDB.upsert("allalbums", newValueDict, controlValueDict)
@@ -348,18 +353,18 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):

                cleanname = helpers.cleanName(artist['artist_name'] + ' ' + rg['title'] + ' ' + track['title'])

-                controlValueDict = {"TrackID":      track['id'],
-                                    "ReleaseID":    rg['id']}
+                controlValueDict = {"TrackID": track['id'],
+                                    "ReleaseID": rg['id']}

-                newValueDict = {"ArtistID":         artistid,
-                                "ArtistName":       artist['artist_name'],
-                                "AlbumTitle":       rg['title'],
-                                "AlbumASIN":        hybridrelease['AlbumASIN'],
-                                "AlbumID":          rg['id'],
-                                "TrackTitle":       track['title'],
-                                "TrackDuration":    track['duration'],
-                                "TrackNumber":      track['number'],
-                                "CleanName":        cleanname
+                newValueDict = {"ArtistID": artistid,
+                                "ArtistName": artist['artist_name'],
+                                "AlbumTitle": rg['title'],
+                                "AlbumASIN": hybridrelease['AlbumASIN'],
+                                "AlbumID": rg['id'],
+                                "TrackTitle": track['title'],
+                                "TrackDuration": track['duration'],
+                                "TrackNumber": track['number'],
+                                "CleanName": cleanname
                            }

                match = myDB.action('SELECT Location, BitRate, Format from have WHERE CleanName=?', [cleanname]).fetchone()
@@ -392,35 +397,35 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):

            album = myDB.action('SELECT * from allalbums WHERE ReleaseID=?', [releaseid]).fetchone()

-            controlValueDict = {"AlbumID":  rg['id']}
+            controlValueDict = {"AlbumID": rg['id']}

-            newValueDict = {"ArtistID":         album['ArtistID'],
-                            "ArtistName":       album['ArtistName'],
-                            "AlbumTitle":       album['AlbumTitle'],
-                            "ReleaseID":        album['ReleaseID'],
-                            "AlbumASIN":        album['AlbumASIN'],
-                            "ReleaseDate":      album['ReleaseDate'],
-                            "Type":             album['Type'],
-                            "ReleaseCountry":   album['ReleaseCountry'],
-                            "ReleaseFormat":    album['ReleaseFormat']
+            newValueDict = {"ArtistID": album['ArtistID'],
+                            "ArtistName": album['ArtistName'],
+                            "AlbumTitle": album['AlbumTitle'],
+                            "ReleaseID": album['ReleaseID'],
+                            "AlbumASIN": album['AlbumASIN'],
+                            "ReleaseDate": album['ReleaseDate'],
+                            "Type": album['Type'],
+                            "ReleaseCountry": album['ReleaseCountry'],
+                            "ReleaseFormat": album['ReleaseFormat']
                        }

            if rg_exists:
                newValueDict['DateAdded'] = rg_exists['DateAdded']
-                newValueDict['Status']    = rg_exists['Status']
+                newValueDict['Status'] = rg_exists['Status']

            else:
                today = helpers.today()

                newValueDict['DateAdded'] = today

-                if headphones.AUTOWANT_ALL:
+                if headphones.CONFIG.AUTOWANT_ALL:
                    newValueDict['Status'] = "Wanted"
-                elif album['ReleaseDate'] > today and headphones.AUTOWANT_UPCOMING:
+                elif album['ReleaseDate'] > today and headphones.CONFIG.AUTOWANT_UPCOMING:
                    newValueDict['Status'] = "Wanted"
                # Sometimes "new" albums are added to musicbrainz after their release date, so let's try to catch these
                # The first test just makes sure we have year-month-day
-                elif helpers.get_age(album['ReleaseDate']) and helpers.get_age(today) - helpers.get_age(album['ReleaseDate']) < 21 and headphones.AUTOWANT_UPCOMING:
+                elif helpers.get_age(album['ReleaseDate']) and helpers.get_age(today) - helpers.get_age(album['ReleaseDate']) < 21 and headphones.CONFIG.AUTOWANT_UPCOMING:
                    newValueDict['Status'] = "Wanted"
                else:
                    newValueDict['Status'] = "Skipped"
@@ -440,21 +445,21 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
                continue

            for track in tracks:
-                controlValueDict = {"TrackID":  track['TrackID'],
-                                    "AlbumID":  rg['id']}
+                controlValueDict = {"TrackID": track['TrackID'],
+                                    "AlbumID": rg['id']}

-                newValueDict = {"ArtistID":     track['ArtistID'],
-                            "ArtistName":       track['ArtistName'],
-                            "AlbumTitle":       track['AlbumTitle'],
-                            "AlbumASIN":        track['AlbumASIN'],
-                            "ReleaseID":        track['ReleaseID'],
-                            "TrackTitle":       track['TrackTitle'],
-                            "TrackDuration":    track['TrackDuration'],
-                            "TrackNumber":      track['TrackNumber'],
-                            "CleanName":        track['CleanName'],
-                            "Location":         track['Location'],
-                            "Format":           track['Format'],
-                            "BitRate":          track['BitRate']
+                newValueDict = {"ArtistID": track['ArtistID'],
+                            "ArtistName": track['ArtistName'],
+                            "AlbumTitle": track['AlbumTitle'],
+                            "AlbumASIN": track['AlbumASIN'],
+                            "ReleaseID": track['ReleaseID'],
+                            "TrackTitle": track['TrackTitle'],
+                            "TrackDuration": track['TrackDuration'],
+                            "TrackNumber": track['TrackNumber'],
+                            "CleanName": track['CleanName'],
+                            "Location": track['Location'],
+                            "Format": track['Format'],
+                            "BitRate": track['BitRate']
                            }

                myDB.upsert("tracks", newValueDict, controlValueDict)
@@ -464,11 +469,11 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
            marked_as_downloaded = False

            if rg_exists:
-                if rg_exists['Status'] == 'Skipped' and ((have_track_count/float(total_track_count)) >= (headphones.ALBUM_COMPLETION_PCT/100.0)):
+                if rg_exists['Status'] == 'Skipped' and ((have_track_count / float(total_track_count)) >= (headphones.CONFIG.ALBUM_COMPLETION_PCT / 100.0)):
                    myDB.action('UPDATE albums SET Status=? WHERE AlbumID=?', ['Downloaded', rg['id']])
                    marked_as_downloaded = True
            else:
-                if ((have_track_count/float(total_track_count)) >= (headphones.ALBUM_COMPLETION_PCT/100.0)):
+                if (have_track_count / float(total_track_count)) >= (headphones.CONFIG.ALBUM_COMPLETION_PCT / 100.0):
                    myDB.action('UPDATE albums SET Status=? WHERE AlbumID=?', ['Downloaded', rg['id']])
                    marked_as_downloaded = True

@@ -478,7 +483,7 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
            # Start a search for the album if it's new, hasn't been marked as
            # downloaded and autowant_all is selected. This search is deferred,
            # in case the search failes and the rest of the import will halt.
-            if not rg_exists and not marked_as_downloaded and headphones.AUTOWANT_ALL:
+            if not rg_exists and not marked_as_downloaded and headphones.CONFIG.AUTOWANT_ALL:
                album_searches.append(rg['id'])
        else:
            if skip_log == 0:
@@ -490,6 +495,9 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
    logger.info(u"Seeing if we need album art for: %s" % artist['artist_name'])
    cache.getThumb(ArtistID=artistid)

+    logger.info(u"Fetching Metacritic reviews for: %s" % artist['artist_name'])
+    metacritic.update(artistid, artist['artist_name'], artist['releasegroups'])
+
    if errors:
        logger.info("[%s] Finished updating artist: %s but with errors, so not marking it as updated in the database" % (artist['artist_name'], artist['artist_name']))
    else:
@@ -504,6 +512,7 @@ def addArtisttoDB(artistid, extrasonly=False, forcefull=False):
        for album_search in album_searches:
            searcher.searchforalbum(albumid=album_search)

+
 def finalize_update(artistid, artistname, errors=False):
    # Moving this little bit to it's own function so we can update have tracks & latest album when deleting extras

@@ -514,25 +523,26 @@ def finalize_update(artistid, artistname, errors=False):
    #havetracks = len(myDB.select('SELECT TrackTitle from tracks WHERE ArtistID=? AND Location IS NOT NULL', [artistid])) + len(myDB.select('SELECT TrackTitle from have WHERE ArtistName like ?', [artist['artist_name']]))
    havetracks = len(myDB.select('SELECT TrackTitle from tracks WHERE ArtistID=? AND Location IS NOT NULL', [artistid])) + len(myDB.select('SELECT TrackTitle from have WHERE ArtistName like ? AND Matched = "Failed"', [artistname]))

-    controlValueDict = {"ArtistID":     artistid}
+    controlValueDict = {"ArtistID": artistid}

    if latestalbum:
-        newValueDict = {"Status":           "Active",
-                        "LatestAlbum":      latestalbum['AlbumTitle'],
-                        "ReleaseDate":      latestalbum['ReleaseDate'],
-                        "AlbumID":          latestalbum['AlbumID'],
-                        "TotalTracks":      totaltracks,
-                        "HaveTracks":       havetracks}
+        newValueDict = {"Status": "Active",
+                        "LatestAlbum": latestalbum['AlbumTitle'],
+                        "ReleaseDate": latestalbum['ReleaseDate'],
+                        "AlbumID": latestalbum['AlbumID'],
+                        "TotalTracks": totaltracks,
+                        "HaveTracks": havetracks}
    else:
-        newValueDict = {"Status":           "Active",
-                        "TotalTracks":      totaltracks,
-                        "HaveTracks":       havetracks}
+        newValueDict = {"Status": "Active",
+                        "TotalTracks": totaltracks,
+                        "HaveTracks": havetracks}

    if not errors:
        newValueDict['LastUpdated'] = helpers.now()

    myDB.upsert("artists", newValueDict, controlValueDict)

+
 def addReleaseById(rid, rgid=None):

    myDB = db.DBConnection()
@@ -543,10 +553,10 @@ def addReleaseById(rid, rgid=None):
        dbalbum = myDB.select("SELECT * from albums WHERE AlbumID=?", [rgid])
        if not dbalbum:
            status = 'Loading'
-            controlValueDict = {"AlbumID":  rgid}
-            newValueDict = {"AlbumTitle":   rgid,
-                            "ArtistName":   status,
-                            "Status":       status}
+            controlValueDict = {"AlbumID": rgid}
+            newValueDict = {"AlbumTitle": rgid,
+                            "ArtistName": status,
+                            "Status": status}
            myDB.upsert("albums", newValueDict, controlValueDict)
            time.sleep(1)

@@ -590,15 +600,26 @@ def addReleaseById(rid, rgid=None):
            sortname = release_dict['artist_name']

        logger.info(u"Now manually adding: " + release_dict['artist_name'] + " - with status Paused")
-        controlValueDict = {"ArtistID":     release_dict['artist_id']}
-        newValueDict = {"ArtistName":       release_dict['artist_name'],
-                        "ArtistSortName":   sortname,
-                        "DateAdded":        helpers.today(),
-                        "Status":           "Paused"}
+        controlValueDict = {"ArtistID": release_dict['artist_id']}
+        newValueDict = {"ArtistName": release_dict['artist_name'],
+                        "ArtistSortName": sortname,
+                        "DateAdded": helpers.today(),
+                        "Status": "Paused"}

-        if headphones.INCLUDE_EXTRAS:
+        if headphones.CONFIG.INCLUDE_EXTRAS:
            newValueDict['IncludeExtras'] = 1
-            newValueDict['Extras'] = headphones.EXTRAS
+            newValueDict['Extras'] = headphones.CONFIG.EXTRAS
+
+        if 'title' in release_dict:
+            newValueDict['LatestAlbum'] = release_dict['title']
+        elif 'rg_title' in release_dict:
+            newValueDict['LatestAlbum'] = release_dict['rg_title']
+
+        if 'date' in release_dict:
+            newValueDict['ReleaseDate'] = release_dict['date']
+
+        if rgid:
+            newValueDict['AlbumID'] = rgid

        myDB.upsert("artists", newValueDict, controlValueDict)

@@ -611,20 +632,20 @@ def addReleaseById(rid, rgid=None):
    if not rg_exists and release_dict or status == 'Loading' and release_dict:  #it should never be the case that we have an rg and not the artist
                                                                                #but if it is this will fail
        logger.info(u"Now adding-by-id album (" + release_dict['title'] + ") from id: " + rgid)
-        controlValueDict = {"AlbumID":  rgid}
+        controlValueDict = {"AlbumID": rgid}
        if status != 'Loading':
            status = 'Wanted'

-        newValueDict = {"ArtistID":         release_dict['artist_id'],
-                        "ReleaseID":        rgid,
-                        "ArtistName":       release_dict['artist_name'],
-                        "AlbumTitle":       release_dict['rg_title'],
-                        "AlbumASIN":        release_dict['asin'],
-                        "ReleaseDate":      release_dict['date'],
-                        "DateAdded":        helpers.today(),
-                        "Status":           status,
-                        "Type":             release_dict['rg_type'],
-                        "ReleaseID":        rid
+        newValueDict = {"ArtistID": release_dict['artist_id'],
+                        "ReleaseID": rgid,
+                        "ArtistName": release_dict['artist_name'],
+                        "AlbumTitle": release_dict['title'] if 'title' in release_dict else release_dict['rg_title'],
+                        "AlbumASIN": release_dict['asin'],
+                        "ReleaseDate": release_dict['date'],
+                        "DateAdded": helpers.today(),
+                        "Status": status,
+                        "Type": release_dict['rg_type'],
+                        "ReleaseID": rid
                        }

        myDB.upsert("albums", newValueDict, controlValueDict)
@@ -635,22 +656,22 @@ def addReleaseById(rid, rgid=None):
        for track in release_dict['tracks']:
            cleanname = helpers.cleanName(release_dict['artist_name'] + ' ' + release_dict['rg_title'] + ' ' + track['title'])

-            controlValueDict = {"TrackID":  track['id'],
-                                "AlbumID":  rgid}
-            newValueDict = {"ArtistID":     release_dict['artist_id'],
-                        "ArtistName":       release_dict['artist_name'],
-                        "AlbumTitle":       release_dict['rg_title'],
-                        "AlbumASIN":        release_dict['asin'],
-                        "TrackTitle":       track['title'],
-                        "TrackDuration":    track['duration'],
-                        "TrackNumber":      track['number'],
-                        "CleanName":        cleanname
+            controlValueDict = {"TrackID": track['id'],
+                                "AlbumID": rgid}
+            newValueDict = {"ArtistID": release_dict['artist_id'],
+                        "ArtistName": release_dict['artist_name'],
+                        "AlbumTitle": release_dict['rg_title'],
+                        "AlbumASIN": release_dict['asin'],
+                        "TrackTitle": track['title'],
+                        "TrackDuration": track['duration'],
+                        "TrackNumber": track['number'],
+                        "CleanName": cleanname
                        }

-            match = myDB.action('SELECT Location, BitRate, Format from have WHERE CleanName=?', [cleanname]).fetchone()
+            match = myDB.action('SELECT Location, BitRate, Format, Matched from have WHERE CleanName=?', [cleanname]).fetchone()

            if not match:
-                match = myDB.action('SELECT Location, BitRate, Format from have WHERE ArtistName LIKE ? AND AlbumTitle LIKE ? AND TrackTitle LIKE ?', [release_dict['artist_name'], release_dict['rg_title'], track['title']]).fetchone()
+                match = myDB.action('SELECT Location, BitRate, Format, Matched from have WHERE ArtistName LIKE ? AND AlbumTitle LIKE ? AND TrackTitle LIKE ?', [release_dict['artist_name'], release_dict['rg_title'], track['title']]).fetchone()

            #if not match:
                #match = myDB.action('SELECT Location, BitRate, Format from have WHERE TrackID=?', [track['id']]).fetchone()
@@ -661,17 +682,26 @@ def addReleaseById(rid, rgid=None):
                newValueDict['Format'] = match['Format']
                #myDB.action('DELETE from have WHERE Location=?', [match['Location']])

+                # If the album has been scanned before adding the release it will be unmatched, update to matched
+                if match['Matched'] == 'Failed':
+                    myDB.action('UPDATE have SET Matched=? WHERE Location=?', (release_dict['rgid'], match['Location']))
+
            myDB.upsert("tracks", newValueDict, controlValueDict)

        # Reset status
        if status == 'Loading':
-            controlValueDict = {"AlbumID":  rgid}
-            newValueDict = {"Status":   "Wanted"}
+            controlValueDict = {"AlbumID": rgid}
+            if headphones.CONFIG.AUTOWANT_MANUALLY_ADDED:
+                newValueDict = {"Status": "Wanted"}
+            else:
+                newValueDict = {"Status": "Skipped"}
            myDB.upsert("albums", newValueDict, controlValueDict)

        # Start a search for the album
-        import searcher
-        searcher.searchforalbum(rgid, False)
+        if headphones.CONFIG.AUTOWANT_MANUALLY_ADDED:
+            import searcher
+            searcher.searchforalbum(rgid, False)
+
    elif not rg_exists and not release_dict:
        logger.error("ReleaseGroup does not exist in the database and did not get a valid response from MB. Skipping release.")
        if status == 'Loading':
@@ -680,6 +710,7 @@ def addReleaseById(rid, rgid=None):
    else:
        logger.info('Release ' + str(rid) + " already exists in the database!")

+
 def updateFormat():
    myDB = db.DBConnection()
    tracks = myDB.select('SELECT * from tracks WHERE Location IS NOT NULL and Format IS NULL')
@@ -688,10 +719,10 @@ def updateFormat():
        for track in tracks:
            try:
                f = MediaFile(track['Location'])
-            except Exception, e:
+            except Exception as e:
                logger.info("Exception from MediaFile for: " + track['Location'] + " : " + str(e))
                continue
-            controlValueDict = {"TrackID":  track['TrackID']}
+            controlValueDict = {"TrackID": track['TrackID']}
            newValueDict = {"Format": f.format}
            myDB.upsert("tracks", newValueDict, controlValueDict)
        logger.info('Finished finding media format for %s files' % len(tracks))
@@ -701,14 +732,15 @@ def updateFormat():
        for track in havetracks:
            try:
                f = MediaFile(track['Location'])
-            except Exception, e:
+            except Exception as e:
                logger.info("Exception from MediaFile for: " + track['Location'] + " : " + str(e))
                continue
-            controlValueDict = {"TrackID":  track['TrackID']}
+            controlValueDict = {"TrackID": track['TrackID']}
            newValueDict = {"Format": f.format}
            myDB.upsert("have", newValueDict, controlValueDict)
        logger.info('Finished finding media format for %s files' % len(havetracks))

+
 def getHybridRelease(fullreleaselist):
    """
    Returns a dictionary of best group of tracks from the list of releases and
@@ -721,18 +753,18 @@ def getHybridRelease(fullreleaselist):
    sortable_release_list = []

    formats = {
-        '2xVinyl':          '2',
-        'Vinyl':            '2',
-        'CD':               '0',
-        'Cassette':         '3',
-        '2xCD':             '1',
-        'Digital Media':    '0'
+        '2xVinyl': '2',
+        'Vinyl': '2',
+        'CD': '0',
+        'Cassette': '3',
+        '2xCD': '1',
+        'Digital Media': '0'
    }

    countries = {
-        'US':    '0',
-        'GB':    '1',
-        'JP':    '2',
+        'US': '0',
+        'GB': '1',
+        'JP': '2',
    }

    for release in fullreleaselist:
@@ -749,14 +781,14 @@ def getHybridRelease(fullreleaselist):

        # Create record
        release_dict = {
-            'hasasin':        bool(release['AlbumASIN']),
-            'asin':           release['AlbumASIN'],
-            'trackscount':    len(release['Tracks']),
-            'releaseid':      release['ReleaseID'],
-            'releasedate':    release['ReleaseDate'],
-            'format':         format,
-            'country':        country,
-            'tracks':         release['Tracks']
+            'hasasin': bool(release['AlbumASIN']),
+            'asin': release['AlbumASIN'],
+            'trackscount': len(release['Tracks']),
+            'releaseid': release['ReleaseID'],
+            'releasedate': release['ReleaseDate'],
+            'format': format,
+            'country': country,
+            'tracks': release['Tracks']
        }

        sortable_release_list.append(release_dict)
@@ -767,8 +799,8 @@ def getHybridRelease(fullreleaselist):
        # Change this value to change the sorting behaviour of none, returning
        # 'None' will put it at the top which was normal behaviour for pre-ngs
        # versions
-        if releaseDate == None:
-            return 'None';
+        if releaseDate is None:
+            return 'None'

        if releaseDate.count('-') == 2:
            return releaseDate
@@ -777,7 +809,7 @@ def getHybridRelease(fullreleaselist):
        else:
            return releaseDate + '13-32'

-    sortable_release_list.sort(key=lambda x:getSortableReleaseDate(x['releasedate']))
+    sortable_release_list.sort(key=lambda x: getSortableReleaseDate(x['releasedate']))

    average_tracks = sum(x['trackscount'] for x in sortable_release_list) / float(len(sortable_release_list))
    for item in sortable_release_list:
@@ -785,9 +817,9 @@ def getHybridRelease(fullreleaselist):

    a = helpers.multikeysort(sortable_release_list, ['-hasasin', 'country', 'format', 'trackscount_delta'])

-    release_dict = {'ReleaseDate'    : sortable_release_list[0]['releasedate'],
-                    'Tracks'         : a[0]['tracks'],
-                    'AlbumASIN'      : a[0]['asin']
+    release_dict = {'ReleaseDate': sortable_release_list[0]['releasedate'],
+                    'Tracks': a[0]['tracks'],
+                    'AlbumASIN': a[0]['asin']
                    }

    return release_dict
@@ -14,17 +14,22 @@
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

 import random
-import time
 import headphones
+import headphones.lock

 from headphones import db, logger, request

 from collections import defaultdict

-TIMEOUT = 60 # seconds
+TIMEOUT = 60.0 # seconds
+REQUEST_LIMIT = 1.0 / 5 # seconds
 ENTRY_POINT = "http://ws.audioscrobbler.com/2.0/"
 API_KEY = "395e6ec6bb557382fc41fde867bce66f"

+# Required for API request limit
+lastfm_lock = headphones.lock.TimedLock(REQUEST_LIMIT)
+
+
 def request_lastfm(method, **kwargs):
    """
    Call a Last.FM API method. Automatically sets the method and API key. Method
@@ -43,7 +48,7 @@ def request_lastfm(method, **kwargs):
    logger.debug("Calling Last.FM method: %s", method)
    logger.debug("Last.FM call parameters: %s", kwargs)

-    data = request.request_json(ENTRY_POINT, timeout=TIMEOUT, params=kwargs)
+    data = request.request_json(ENTRY_POINT, timeout=TIMEOUT, params=kwargs, lock=lastfm_lock)

    # Parse response and check for errors.
    if not data:
@@ -56,6 +61,7 @@ def request_lastfm(method, **kwargs):

    return data

+
 def getSimilar():
    myDB = db.DBConnection()
    results = myDB.select("SELECT ArtistID from artists ORDER BY HaveTracks DESC")
@@ -65,7 +71,6 @@ def getSimilar():

    for result in results[:12]:
        data = request_lastfm("artist.getsimilar", mbid=result["ArtistId"])
-        time.sleep(10)

        if data and "similarartists" in data:
            artists = data["similarartists"]["artist"]
@@ -101,16 +106,17 @@ def getSimilar():

    logger.debug("Inserted %d artists into Last.FM tag cloud", len(top_list))

+
 def getArtists():
    myDB = db.DBConnection()
    results = myDB.select("SELECT ArtistID from artists")

-    if not headphones.LASTFM_USERNAME:
+    if not headphones.CONFIG.LASTFM_USERNAME:
        logger.warn("Last.FM username not set, not importing artists.")
        return

-    logger.info("Fetching artists from Last.FM for username: %s", headphones.LASTFM_USERNAME)
-    data = request_lastfm("library.getartists", limit=10000, user=headphones.LASTFM_USERNAME)
+    logger.info("Fetching artists from Last.FM for username: %s", headphones.CONFIG.LASTFM_USERNAME)
+    data = request_lastfm("library.getartists", limit=10000, user=headphones.CONFIG.LASTFM_USERNAME)

    if data and "artists" in data:
        artistlist = []
@@ -130,6 +136,7 @@ def getArtists():

        logger.info("Imported %d new artists from Last.FM", len(artistlist))

+
 def getTagTopArtists(tag, limit=50):
    myDB = db.DBConnection()
    results = myDB.select("SELECT ArtistID from artists")
@@ -153,4 +160,4 @@ def getTagTopArtists(tag, limit=50):
        for artistid in artistlist:
            importer.addArtisttoDB(artistid)

-        logger.debug("Added %d new artists from Last.FM", len(artistlist))
+        logger.debug("Added %d new artists from Last.FM", len(artistlist))
@@ -14,25 +14,25 @@
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

 import os
-import glob
 import headphones

 from beets.mediafile import MediaFile, FileTypeError, UnreadableFileError

 from headphones import db, logger, helpers, importer, lastfm

-# You can scan a single directory and append it to the current library by specifying append=True, ArtistID & ArtistName
-def libraryScan(dir=None, append=False, ArtistID=None, ArtistName=None, cron=False):
+# You can scan a single directory and append it to the current library by
+# specifying append=True, ArtistID and ArtistName.
+def libraryScan(dir=None, append=False, ArtistID=None, ArtistName=None,
+    cron=False):

-
-    if cron and not headphones.LIBRARYSCAN:
+    if cron and not headphones.CONFIG.LIBRARYSCAN:
        return

    if not dir:
-        if not headphones.MUSIC_DIR:
+        if not headphones.CONFIG.MUSIC_DIR:
            return
        else:
-            dir = headphones.MUSIC_DIR
+            dir = headphones.CONFIG.MUSIC_DIR

    # If we're appending a dir, it's coming from the post processor which is
    # already bytestring
@@ -64,36 +64,33 @@ def libraryScan(dir=None, append=False, ArtistID=None, ArtistName=None, cron=Fal
            encoded_track_string = track['Location'].encode(headphones.SYS_ENCODING, 'replace')
            if not os.path.isfile(encoded_track_string):
                if track['ArtistName']:
-                    #Make sure deleted files get accounted for when updating artist track counts
+                    # Make sure deleted files get accounted for when updating artist track counts
                    new_artists.append(track['ArtistName'])
                myDB.action('DELETE FROM have WHERE Location=?', [track['Location']])
                logger.info('File %s removed from Headphones, as it is no longer on disk' % encoded_track_string.decode(headphones.SYS_ENCODING, 'replace'))
-           ###############myDB.action('DELETE from have')

    bitrates = []
-
    song_list = []
+    latest_subdirectory = []
+
    new_song_count = 0
    file_count = 0

-    latest_subdirectory = []
-
-    for r,d,f in os.walk(dir):
-        #need to abuse slicing to get a copy of the list, doing it directly will skip the element after a deleted one
-        #using a list comprehension will not work correctly for nested subdirectories (os.walk keeps its original list)
-        for directory in d[:]:
-            if directory.startswith("."):
-                d.remove(directory)
+    for r, d, f in helpers.walk_directory(dir):
+        # Filter paths based on config. Note that these methods work directly
+        # on the inputs
+        helpers.path_filter_patterns(d, headphones.CONFIG.IGNORED_FOLDERS, r)
+        helpers.path_filter_patterns(f, headphones.CONFIG.IGNORED_FILES, r)

        for files in f:
            # MEDIA_FORMATS = music file extensions, e.g. mp3, flac, etc
            if any(files.lower().endswith('.' + x.lower()) for x in headphones.MEDIA_FORMATS):
-
-                subdirectory = r.replace(dir,'')
+                subdirectory = r.replace(dir, '')
                latest_subdirectory.append(subdirectory)
-                if file_count == 0 and r.replace(dir,'') !='':
+
+                if file_count == 0 and r.replace(dir, '') != '':
                    logger.info("[%s] Now scanning subdirectory %s" % (dir.decode(headphones.SYS_ENCODING, 'replace'), subdirectory.decode(headphones.SYS_ENCODING, 'replace')))
-                elif latest_subdirectory[file_count] != latest_subdirectory[file_count-1] and file_count !=0:
+                elif latest_subdirectory[file_count] != latest_subdirectory[file_count - 1] and file_count != 0:
                    logger.info("[%s] Now scanning subdirectory %s" % (dir.decode(headphones.SYS_ENCODING, 'replace'), subdirectory.decode(headphones.SYS_ENCODING, 'replace')))

                song = os.path.join(r, files)
@@ -105,7 +102,10 @@ def libraryScan(dir=None, append=False, ArtistID=None, ArtistName=None, cron=Fal
                try:
                    f = MediaFile(song)
                except (FileTypeError, UnreadableFileError):
-                    logger.error("Cannot read file media file '%s'. It may be corrupted or not a media file.", unicode_song_path)
+                    logger.warning("Cannot read media file '%s', skipping. It may be corrupted or not a media file.", unicode_song_path)
+                    continue
+                except IOError:
+                    logger.warning("Cannnot read media file '%s', skipping. Does the file exists?", unicode_song_path)
                    continue

                # Grab the bitrates for the auto detect bit rate option
@@ -124,24 +124,24 @@ def libraryScan(dir=None, append=False, ArtistID=None, ArtistName=None, cron=Fal
                # TODO: skip adding songs without the minimum requisite information (just a matter of putting together the right if statements)

                if f_artist and f.album and f.title:
-                    CleanName = helpers.cleanName(f_artist +' '+ f.album +' '+ f.title)
+                    CleanName = helpers.cleanName(f_artist + ' ' + f.album + ' ' + f.title)
                else:
                    CleanName = None

-                controlValueDict = {'Location'   : unicode_song_path}
+                controlValueDict = {'Location': unicode_song_path}

-                newValueDict = { 'TrackID' : f.mb_trackid,
+                newValueDict = {'TrackID': f.mb_trackid,
                              #'ReleaseID' : f.mb_albumid,
-                              'ArtistName' : f_artist,
-                              'AlbumTitle' : f.album,
+                              'ArtistName': f_artist,
+                              'AlbumTitle': f.album,
                              'TrackNumber': f.track,
                              'TrackLength': f.length,
-                              'Genre'      : f.genre,
-                              'Date'       : f.date,
-                              'TrackTitle' : f.title,
-                              'BitRate'    : f.bitrate,
-                              'Format'     : f.format,
-                              'CleanName'  : CleanName
+                              'Genre': f.genre,
+                              'Date': f.date,
+                              'TrackTitle': f.title,
+                              'BitRate': f.bitrate,
+                              'Format': f.format,
+                              'CleanName': CleanName
                              }

                #song_list.append(song_dict)
@@ -152,7 +152,7 @@ def libraryScan(dir=None, append=False, ArtistID=None, ArtistName=None, cron=Fal
                    if f_artist:
                        new_artists.append(f_artist)
                    myDB.upsert("have", newValueDict, controlValueDict)
-                    new_song_count+=1
+                    new_song_count += 1
                else:
                    if check_exist_song['ArtistName'] != f_artist or check_exist_song['AlbumTitle'] != f.album or check_exist_song['TrackTitle'] != f.title:
                        #Important track metadata has been modified, need to run matcher again
@@ -167,19 +167,18 @@ def libraryScan(dir=None, append=False, ArtistID=None, ArtistName=None, cron=Fal
                        myDB.upsert("have", newValueDict, controlValueDict)
                        myDB.action('UPDATE tracks SET Location=?, BitRate=?, Format=? WHERE Location=?', [None, None, None, unicode_song_path])
                        myDB.action('UPDATE alltracks SET Location=?, BitRate=?, Format=? WHERE Location=?', [None, None, None, unicode_song_path])
-                        new_song_count+=1
+                        new_song_count += 1
                    else:
                        #This track information hasn't changed
                        if f_artist and check_exist_song['Matched'] != "Ignored":
                            new_artists.append(f_artist)

-                file_count+=1
-
+                file_count += 1

    # Now we start track matching
    logger.info("%s new/modified songs found and added to the database" % new_song_count)
-    song_list = myDB.action("SELECT * FROM have WHERE Matched IS NULL AND LOCATION LIKE ?", [dir.decode(headphones.SYS_ENCODING, 'replace')+"%"])
-    total_number_of_songs = myDB.action("SELECT COUNT(*) FROM have WHERE Matched IS NULL AND LOCATION LIKE ?", [dir.decode(headphones.SYS_ENCODING, 'replace')+"%"]).fetchone()[0]
+    song_list = myDB.action("SELECT * FROM have WHERE Matched IS NULL AND LOCATION LIKE ?", [dir.decode(headphones.SYS_ENCODING, 'replace') + "%"])
+    total_number_of_songs = myDB.action("SELECT COUNT(*) FROM have WHERE Matched IS NULL AND LOCATION LIKE ?", [dir.decode(headphones.SYS_ENCODING, 'replace') + "%"]).fetchone()[0]
    logger.info("Found " + str(total_number_of_songs) + " new/modified tracks in: '" + dir.decode(headphones.SYS_ENCODING, 'replace') + "'. Matching tracks to the appropriate releases....")

    # Sort the song_list by most vague (e.g. no trackid or releaseid) to most specific (both trackid & releaseid)
@@ -197,14 +196,13 @@ def libraryScan(dir=None, append=False, ArtistID=None, ArtistName=None, cron=Fal
        latest_artist.append(song['ArtistName'])
        if song_count == 0:
            logger.info("Now matching songs by %s" % song['ArtistName'])
-        elif latest_artist[song_count] != latest_artist[song_count-1] and song_count !=0:
+        elif latest_artist[song_count] != latest_artist[song_count - 1] and song_count != 0:
            logger.info("Now matching songs by %s" % song['ArtistName'])

-        #print song['ArtistName']+' - '+song['AlbumTitle']+' - '+song['TrackTitle']
        song_count += 1
-        completion_percentage = float(song_count)/total_number_of_songs * 100
+        completion_percentage = float(song_count) / total_number_of_songs * 100

-        if completion_percentage%10 == 0:
+        if completion_percentage % 10 == 0:
            logger.info("Track matching is " + str(completion_percentage) + "% complete")

        #THE "MORE-SPECIFIC" CLAUSES HERE HAVE ALL BEEN REMOVED.  WHEN RUNNING A LIBRARY SCAN, THE ONLY CLAUSES THAT
@@ -217,79 +215,78 @@ def libraryScan(dir=None, append=False, ArtistID=None, ArtistName=None, cron=Fal
            track = myDB.action('SELECT ArtistName, AlbumTitle, TrackTitle, AlbumID from tracks WHERE ArtistName LIKE ? AND AlbumTitle LIKE ? AND TrackTitle LIKE ?', [song['ArtistName'], song['AlbumTitle'], song['TrackTitle']]).fetchone()
            have_updated = False
            if track:
-                controlValueDict = { 'ArtistName' : track['ArtistName'],
-                                     'AlbumTitle' : track['AlbumTitle'],
-                                     'TrackTitle' : track['TrackTitle'] }
-                newValueDict = { 'Location' : song['Location'],
-                                 'BitRate'  : song['BitRate'],
-                                 'Format'   : song['Format'] }
+                controlValueDict = {'ArtistName': track['ArtistName'],
+                                     'AlbumTitle': track['AlbumTitle'],
+                                     'TrackTitle': track['TrackTitle']}
+                newValueDict = {'Location': song['Location'],
+                                 'BitRate': song['BitRate'],
+                                 'Format': song['Format']}
                myDB.upsert("tracks", newValueDict, controlValueDict)

-                controlValueDict2 = { 'Location' : song['Location']}
-                newValueDict2 = { 'Matched' : track['AlbumID']}
+                controlValueDict2 = {'Location': song['Location']}
+                newValueDict2 = {'Matched': track['AlbumID']}
                myDB.upsert("have", newValueDict2, controlValueDict2)
                have_updated = True
            else:
                track = myDB.action('SELECT CleanName, AlbumID from tracks WHERE CleanName LIKE ?', [song['CleanName']]).fetchone()
                if track:
-                    controlValueDict = { 'CleanName' : track['CleanName']}
-                    newValueDict = { 'Location' : song['Location'],
-                                     'BitRate'  : song['BitRate'],
-                                     'Format'   : song['Format'] }
+                    controlValueDict = {'CleanName': track['CleanName']}
+                    newValueDict = {'Location': song['Location'],
+                                     'BitRate': song['BitRate'],
+                                     'Format': song['Format']}
                    myDB.upsert("tracks", newValueDict, controlValueDict)

-                    controlValueDict2 = { 'Location' : song['Location']}
-                    newValueDict2 = { 'Matched' : track['AlbumID']}
+                    controlValueDict2 = {'Location': song['Location']}
+                    newValueDict2 = {'Matched': track['AlbumID']}
                    myDB.upsert("have", newValueDict2, controlValueDict2)
                    have_updated = True
                else:
-                    controlValueDict2 = { 'Location' : song['Location']}
-                    newValueDict2 = { 'Matched' : "Failed"}
+                    controlValueDict2 = {'Location': song['Location']}
+                    newValueDict2 = {'Matched': "Failed"}
                    myDB.upsert("have", newValueDict2, controlValueDict2)
                    have_updated = True

            alltrack = myDB.action('SELECT ArtistName, AlbumTitle, TrackTitle, AlbumID from alltracks WHERE ArtistName LIKE ? AND AlbumTitle LIKE ? AND TrackTitle LIKE ?', [song['ArtistName'], song['AlbumTitle'], song['TrackTitle']]).fetchone()
            if alltrack:
-                controlValueDict = { 'ArtistName' : alltrack['ArtistName'],
-                                     'AlbumTitle' : alltrack['AlbumTitle'],
-                                     'TrackTitle' : alltrack['TrackTitle'] }
-                newValueDict = { 'Location' : song['Location'],
-                                 'BitRate'  : song['BitRate'],
-                                 'Format'   : song['Format'] }
+                controlValueDict = {'ArtistName': alltrack['ArtistName'],
+                                     'AlbumTitle': alltrack['AlbumTitle'],
+                                     'TrackTitle': alltrack['TrackTitle']}
+                newValueDict = {'Location': song['Location'],
+                                 'BitRate': song['BitRate'],
+                                 'Format': song['Format']}
                myDB.upsert("alltracks", newValueDict, controlValueDict)

-                controlValueDict2 = { 'Location' : song['Location']}
-                newValueDict2 = { 'Matched' : alltrack['AlbumID']}
+                controlValueDict2 = {'Location': song['Location']}
+                newValueDict2 = {'Matched': alltrack['AlbumID']}
                myDB.upsert("have", newValueDict2, controlValueDict2)
            else:
                alltrack = myDB.action('SELECT CleanName, AlbumID from alltracks WHERE CleanName LIKE ?', [song['CleanName']]).fetchone()
                if alltrack:
-                    controlValueDict = { 'CleanName' : alltrack['CleanName']}
-                    newValueDict = { 'Location' : song['Location'],
-                                     'BitRate'  : song['BitRate'],
-                                     'Format'   : song['Format'] }
+                    controlValueDict = {'CleanName': alltrack['CleanName']}
+                    newValueDict = {'Location': song['Location'],
+                                     'BitRate': song['BitRate'],
+                                     'Format': song['Format']}
                    myDB.upsert("alltracks", newValueDict, controlValueDict)

-                    controlValueDict2 = { 'Location' : song['Location']}
-                    newValueDict2 = { 'Matched' : alltrack['AlbumID']}
+                    controlValueDict2 = {'Location': song['Location']}
+                    newValueDict2 = {'Matched': alltrack['AlbumID']}
                    myDB.upsert("have", newValueDict2, controlValueDict2)
                else:
                    # alltracks may not exist if adding album manually, have should only be set to failed if not already updated in tracks
                    if not have_updated:
-                        controlValueDict2 = { 'Location' : song['Location']}
-                        newValueDict2 = { 'Matched' : "Failed"}
+                        controlValueDict2 = {'Location': song['Location']}
+                        newValueDict2 = {'Matched': "Failed"}
                        myDB.upsert("have", newValueDict2, controlValueDict2)

        else:
-            controlValueDict2 = { 'Location' : song['Location']}
-            newValueDict2 = { 'Matched' : "Failed"}
+            controlValueDict2 = {'Location': song['Location']}
+            newValueDict2 = {'Matched': "Failed"}
            myDB.upsert("have", newValueDict2, controlValueDict2)

        #######myDB.action('INSERT INTO have (ArtistName, AlbumTitle, TrackNumber, TrackTitle, TrackLength, BitRate, Genre, Date, TrackID, Location, CleanName, Format) VALUES( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)', [song['ArtistName'], song['AlbumTitle'], song['TrackNumber'], song['TrackTitle'], song['TrackLength'], song['BitRate'], song['Genre'], song['Date'], song['TrackID'], song['Location'], CleanName, song['Format']])

    logger.info('Completed matching tracks from directory: %s' % dir.decode(headphones.SYS_ENCODING, 'replace'))

-
    if not append:
        logger.info('Updating scanned artist track counts')

@@ -298,23 +295,37 @@ def libraryScan(dir=None, append=False, ArtistID=None, ArtistName=None, cron=Fal
        current_artists = myDB.select('SELECT ArtistName, ArtistID from artists')

        #There was a bug where artists with special characters (-,') would show up in new artists.
-        artist_list = [f for f in unique_artists if helpers.cleanName(f).lower() not in [helpers.cleanName(x[0]).lower() for x in current_artists]]
-        artists_checked = [f for f in unique_artists if helpers.cleanName(f).lower() in [helpers.cleanName(x[0]).lower() for x in current_artists]]
+        artist_list = [
+            x for x in unique_artists
+            if helpers.cleanName(x).lower() not in [
+                helpers.cleanName(y[0]).lower()
+                for y in current_artists
+            ]
+        ]
+        artists_checked = [
+            x for x in unique_artists
+            if helpers.cleanName(x).lower() in [
+                helpers.cleanName(y[0]).lower()
+                for y in current_artists
+            ]
+        ]

        # Update track counts
-
        for artist in artists_checked:
            # Have tracks are selected from tracks table and not all tracks because of duplicates
            # We update the track count upon an album switch to compliment this
-            havetracks = len(myDB.select('SELECT TrackTitle from tracks WHERE ArtistName like ? AND Location IS NOT NULL', [artist])) + len(myDB.select('SELECT TrackTitle from have WHERE ArtistName like ? AND Matched = "Failed"', [artist]))
-            #Note, some people complain about having "artist have tracks" > # of tracks total in artist official releases
+            havetracks = (
+                len(myDB.select('SELECT TrackTitle from tracks WHERE ArtistName like ? AND Location IS NOT NULL', [artist]))
+                + len(myDB.select('SELECT TrackTitle from have WHERE ArtistName like ? AND Matched = "Failed"', [artist]))
+            )
+            # Note: some people complain about having "artist have tracks" > # of tracks total in artist official releases
            # (can fix by getting rid of second len statement)
            myDB.action('UPDATE artists SET HaveTracks=? WHERE ArtistName=?', [havetracks, artist])

        logger.info('Found %i new artists' % len(artist_list))

-        if len(artist_list):
-            if headphones.ADD_ARTISTS:
+        if artist_list:
+            if headphones.CONFIG.AUTO_ADD_ARTISTS:
                logger.info('Importing %i new artists' % len(artist_list))
                importer.artistlist_to_mbids(artist_list)
            else:
@@ -323,8 +334,8 @@ def libraryScan(dir=None, append=False, ArtistID=None, ArtistName=None, cron=Fal
                for artist in artist_list:
                    myDB.action('INSERT OR IGNORE INTO newartists VALUES (?)', [artist])

-        if headphones.DETECT_BITRATE:
-            headphones.PREFERRED_BITRATE = sum(bitrates)/len(bitrates)/1000
+        if headphones.CONFIG.DETECT_BITRATE:
+            headphones.CONFIG.PREFERRED_BITRATE = sum(bitrates) / len(bitrates) / 1000

    else:
        # If we're appending a new album to the database, update the artists total track counts
@@ -336,9 +347,12 @@ def libraryScan(dir=None, append=False, ArtistID=None, ArtistName=None, cron=Fal
    if not append:
        update_album_status()
        lastfm.getSimilar()
+
    logger.info('Library scan complete')

 #ADDED THIS SECTION TO MARK ALBUMS AS DOWNLOADED IF ARTISTS ARE ADDED EN MASSE BEFORE LIBRARY IS SCANNED
+
+
 def update_album_status(AlbumID=None):
    myDB = db.DBConnection()
    logger.info('Counting matched tracks to mark albums as skipped/downloaded')
@@ -351,16 +365,16 @@ def update_album_status(AlbumID=None):
        total_tracks = 0
        have_tracks = 0
        for track in track_counter:
-            total_tracks+=1
+            total_tracks += 1
            if track['Location']:
-                have_tracks+=1
+                have_tracks += 1
        if total_tracks != 0:
            album_completion = float(float(have_tracks) / float(total_tracks)) * 100
        else:
            album_completion = 0
            logger.info('Album %s does not have any tracks in database' % album['AlbumTitle'])

-        if album_completion >= headphones.ALBUM_COMPLETION_PCT and album['Status'] == 'Skipped':
+        if album_completion >= headphones.CONFIG.ALBUM_COMPLETION_PCT and album['Status'] == 'Skipped':
            new_album_status = "Downloaded"

        # I don't think we want to change Downloaded->Skipped.....
@@ -375,7 +389,7 @@ def update_album_status(AlbumID=None):
        else:
            new_album_status = album['Status']

-        myDB.upsert("albums", {'Status'   : new_album_status}, {'AlbumID'   : album['AlbumID']})
+        myDB.upsert("albums", {'Status': new_album_status}, {'AlbumID': album['AlbumID']})
        if new_album_status != album['Status']:
            logger.info('Album %s changed to %s' % (album['AlbumTitle'], new_album_status))
    logger.info('Album status update complete')
@@ -0,0 +1,87 @@
+"""
+Locking-related classes
+"""
+
+import headphones.logger
+import time
+import threading
+import Queue
+
+
+class TimedLock(object):
+    """
+    Enforce request rate limit if applicable. This uses the lock so there
+    is synchronized access to the API. When N threads enter this method, the
+    first will pass trough, since there there was no last request recorded.
+    The last request time will be set. Then, the second thread will unlock,
+    and see that the last request was X seconds ago. It will sleep
+    (request_limit - X) seconds, and then continue. Then the third one will
+    unblock, and so on. After all threads finish, the total time will at
+    least be (N * request_limit) seconds. If some request takes longer than
+    request_limit seconds, the next unblocked thread will wait less.
+    """
+
+    def __init__(self, minimum_delta=0):
+        """
+        Set up the lock
+        """
+        self.lock = threading.Lock()
+        self.last_used = 0
+        self.minimum_delta = minimum_delta
+        self.queue = Queue.Queue()
+
+    def __enter__(self):
+        """
+        Called when with lock: is invoked
+        """
+        self.lock.acquire()
+        delta = time.time() - self.last_used
+        sleep_amount = self.minimum_delta - delta
+        if sleep_amount >= 0:
+            # zero sleeps give the cpu a chance to task-switch
+            headphones.logger.debug('Sleeping %s (interval)', sleep_amount)
+            time.sleep(sleep_amount)
+        while not self.queue.empty():
+            try:
+                seconds = self.queue.get(False)
+                headphones.logger.debug('Sleeping %s (queued)', seconds)
+                time.sleep(seconds)
+            except Queue.Empty:
+                continue
+            self.queue.task_done()
+
+    def __exit__(self, type, value, traceback):
+        """
+        Called when exiting the with block.
+        """
+        self.last_used = time.time()
+        self.lock.release()
+
+    def snooze(self, seconds):
+        """
+        Asynchronously add time to the next request. Can be called outside
+        of the lock context, but it is possible for the next lock holder
+        to not check the queue until after something adds time to it.
+        """
+        # We use a queue so that we don't have to synchronize
+        # across threads and with or without locks
+        headphones.logger.info('Adding %s to queue', seconds)
+        self.queue.put(seconds)
+
+
+class FakeLock(object):
+    """
+    If no locking or request throttling is needed, use this
+    """
+
+    def __enter__(self):
+        """
+        Do nothing on enter
+        """
+        pass
+
+    def __exit__(self, type, value, traceback):
+        """
+        Do nothing on exit
+        """
+        pass
@@ -24,6 +24,7 @@ import headphones
 import threading
 import traceback
 import logging
+import errno
 import sys
 import os

@@ -35,8 +36,8 @@ MAX_FILES = 5
 # Headphones logger
 logger = logging.getLogger("headphones")

-# Global queue of multiprocessing logging
-queue = multiprocessing.Queue()
+# Global queue for multiprocessing logging
+queue = None

 class LogListHandler(logging.Handler):
    """
@@ -49,6 +50,7 @@ class LogListHandler(logging.Handler):

        headphones.LOG_LIST.insert(0, (helpers.now(), message, record.levelname, record.threadName))

+
@contextlib.contextmanager
 def listener():
    """
@@ -56,13 +58,34 @@ def listener():
    To be used in a with statement in the main process, for multiprocessing.
    """

-    queue_listener = QueueListener(queue, *logger.handlers)
+    global queue

-    try:
-        queue_listener.start()
+    # Initialize queue if not already done
+    if queue is None:
+        try:
+            queue = multiprocessing.Queue()
+        except OSError as e:
+            queue = False
+
+            # Some machines don't have access to /dev/shm. See
+            # http://stackoverflow.com/questions/2009278 for more information.
+            if e.errno == errno.EACCES:
+                logger.warning("Multiprocess logging disabled, because "
+                    "current user cannot map shared memory. You won't see any" \
+                    "logging generated by the worker processed.")
+
+    # Multiprocess logging may be disabled.
+    if not queue:
        yield
-    finally:
-        queue_listener.stop()
+    else:
+        queue_listener = QueueListener(queue, *logger.handlers)
+
+        try:
+            queue_listener.start()
+            yield
+        finally:
+            queue_listener.stop()
+

 def initMultiprocessing():
    """
@@ -71,6 +94,11 @@ def initMultiprocessing():
    completely.
    """

+    # Multiprocess logging may be disabled.
+    if not queue:
+        return
+
+    # Remove all handlers and add the Queue handler as the only one.
    for handler in logger.handlers[:]:
        logger.removeHandler(handler)

@@ -82,7 +110,8 @@ def initMultiprocessing():
    # Change current thread name for log record
    threading.current_thread().name = multiprocessing.current_process().name

-def initLogger(console=False, verbose=False):
+
+def initLogger(console=False, log_dir=False, verbose=False):
    """
    Setup logging for Headphones. It uses the logger instance with the name
    'headphones'. Three log handlers are added:
@@ -91,7 +120,8 @@ def initLogger(console=False, verbose=False):
    * LogListHandler: for Web UI
    * StreamHandler: for console (if console)

-    Console logging is only enabled if console is set to True.
+    Console logging is only enabled if console is set to True. This method can
+    be invoked multiple times, during different stages of Headphones.
    """

    # Close and remove old handlers. This is required to reinit the loggers
@@ -109,22 +139,23 @@ def initLogger(console=False, verbose=False):
    logger.propagate = False
    logger.setLevel(logging.DEBUG if verbose else logging.INFO)

-    # Setup file logger
-    filename = os.path.join(headphones.LOG_DIR, FILENAME)
-
-    file_formatter = logging.Formatter('%(asctime)s - %(levelname)-7s :: %(threadName)s : %(message)s', '%d-%b-%Y %H:%M:%S')
-    file_handler = handlers.RotatingFileHandler(filename, maxBytes=MAX_SIZE, backupCount=MAX_FILES)
-    file_handler.setLevel(logging.DEBUG)
-    file_handler.setFormatter(file_formatter)
-
-    logger.addHandler(file_handler)
-
    # Add list logger
    loglist_handler = LogListHandler()
    loglist_handler.setLevel(logging.DEBUG)

    logger.addHandler(loglist_handler)

+    # Setup file logger
+    if log_dir:
+        filename = os.path.join(log_dir, FILENAME)
+
+        file_formatter = logging.Formatter('%(asctime)s - %(levelname)-7s :: %(threadName)s : %(message)s', '%d-%b-%Y %H:%M:%S')
+        file_handler = handlers.RotatingFileHandler(filename, maxBytes=MAX_SIZE, backupCount=MAX_FILES)
+        file_handler.setLevel(logging.DEBUG)
+        file_handler.setFormatter(file_formatter)
+
+        logger.addHandler(file_handler)
+
    # Setup console logger
    if console:
        console_formatter = logging.Formatter('%(asctime)s - %(levelname)s :: %(threadName)s : %(message)s', '%d-%b-%Y %H:%M:%S')
@@ -137,6 +168,7 @@ def initLogger(console=False, verbose=False):
    # Install exception hooks
    initHooks()

+
 def initHooks(global_exceptions=True, thread_exceptions=True, pass_original=True):
    """
    This method installs exception catching mechanisms. Any exception caught
@@ -191,4 +223,4 @@ warn = logger.warn
 error = logger.error
 debug = logger.debug
 warning = logger.warning
-exception = logger.exception
+exception = logger.exception
@@ -18,9 +18,10 @@ import htmlentitydefs

 from headphones import logger, request

+
 def getLyrics(artist, song):

-    params = {  "artist": artist.encode('utf-8'),
+    params = {"artist": artist.encode('utf-8'),
                "song": song.encode('utf-8'),
                "fmt": 'xml'
                }
@@ -60,6 +61,7 @@ def getLyrics(artist, song):

    return lyrics

+
 def convert_html_entities(s):
    matches = re.findall("&#\d+;", s)
    if len(matches) > 0:
@@ -79,7 +81,7 @@ def convert_html_entities(s):
        hits.remove(amp)
    for hit in hits:
        name = hit[1:-1]
-        if htmlentitydefs.name2codepoint.has_key(name):
-                s = s.replace(hit, unichr(htmlentitydefs.name2codepoint[name]))
+        if name in htmlentitydefs.name2codepoint:
+            s = s.replace(hit, unichr(htmlentitydefs.name2codepoint[name]))
    s = s.replace(amp, "&")
    return s
@@ -15,404 +15,467 @@


 from headphones import logger, db, helpers
-from headphones.helpers import multikeysort, replace_all

-import time
-import threading
 import headphones
 import musicbrainzngs
+import headphones.lock

 try:
+    # pylint:disable=E0611
+    # ignore this error because we are catching the ImportError
    from collections import OrderedDict
+    # pylint:enable=E0611
 except ImportError:
    # Python 2.6.x fallback, from libs
-    import OrderedDict
+    from ordereddict import OrderedDict

-mb_lock = threading.Lock()
+mb_lock = headphones.lock.TimedLock(0)

 # Quick fix to add mirror switching on the fly. Need to probably return the mbhost & mbport that's
 # being used, so we can send those values to the log
+
+
 def startmb():

    mbuser = None
    mbpass = None

-    if headphones.MIRROR == "musicbrainz.org":
+    if headphones.CONFIG.MIRROR == "musicbrainz.org":
        mbhost = "musicbrainz.org"
        mbport = 80
        sleepytime = 1
-    elif headphones.MIRROR == "custom":
-        mbhost = headphones.CUSTOMHOST
-        mbport = int(headphones.CUSTOMPORT)
-        sleepytime = int(headphones.CUSTOMSLEEP)
-    elif headphones.MIRROR == "headphones":
+    elif headphones.CONFIG.MIRROR == "custom":
+        mbhost = headphones.CONFIG.CUSTOMHOST
+        mbport = int(headphones.CONFIG.CUSTOMPORT)
+        mbuser = headphones.CONFIG.CUSTOMUSER
+        mbpass = headphones.CONFIG.CUSTOMPASS
+        sleepytime = int(headphones.CONFIG.CUSTOMSLEEP)
+    elif headphones.CONFIG.MIRROR == "headphones":
        mbhost = "144.76.94.239"
        mbport = 8181
-        mbuser = headphones.HPUSER
-        mbpass = headphones.HPPASS
+        mbuser = headphones.CONFIG.HPUSER
+        mbpass = headphones.CONFIG.HPPASS
        sleepytime = 0
    else:
        return False

-    musicbrainzngs.set_useragent("headphones","0.0","https://github.com/rembo10/headphones")
+    musicbrainzngs.set_useragent("headphones", "0.0", "https://github.com/rembo10/headphones")
    musicbrainzngs.set_hostname(mbhost + ":" + str(mbport))
+
+    # Their rate limiting should be redundant to our lock
    if sleepytime == 0:
        musicbrainzngs.set_rate_limit(False)
    else:
        #calling it with an it ends up blocking all requests after the first
        musicbrainzngs.set_rate_limit(limit_or_interval=float(sleepytime))
+        mb_lock.minimum_delta = sleepytime

    # Add headphones credentials
-    if headphones.MIRROR == "headphones":
-        if not mbuser and mbpass:
-            logger.warn("No username or password set for VIP server")
+    if headphones.CONFIG.MIRROR == "headphones" or headphones.CONFIG.CUSTOMAUTH:
+        if not mbuser or not mbpass:
+            logger.warn("No username or password set for MusicBrainz server")
        else:
-            musicbrainzngs.hpauth(mbuser,mbpass)
+            musicbrainzngs.hpauth(mbuser, mbpass)
+
+    # Let us know if we disable custom authentication
+    if not headphones.CONFIG.CUSTOMAUTH and headphones.CONFIG.MIRROR == "custom":
+        musicbrainzngs.disable_hpauth()

    logger.debug('Using the following server values: MBHost: %s, MBPort: %i, Sleep Interval: %i', mbhost, mbport, sleepytime)

    return True

+
 def findArtist(name, limit=1):
+     artistlist = []
+     artistResults = None

-    with mb_lock:
-        artistlist = []
-        artistResults = None
+     chars = set('!?*-')
+     if any((c in chars) for c in name):
+         name = '"' + name + '"'

-        chars = set('!?*-')
-        if any((c in chars) for c in name):
-            name = '"'+name+'"'
+     criteria = {'artist': name.lower()}

-        criteria = {'artist': name.lower()}
+     with mb_lock:
+         try:
+             artistResults = musicbrainzngs.search_artists(limit=limit, **criteria)['artist-list']
+         except musicbrainzngs.WebServiceError as e:
+             logger.warn('Attempt to query MusicBrainz for %s failed (%s)' % (name, str(e)))
+             mb_lock.snooze(5)

-        try:
-            artistResults = musicbrainzngs.search_artists(limit=limit, **criteria)['artist-list']
-        except musicbrainzngs.WebServiceError as e:
-            logger.warn('Attempt to query MusicBrainz for %s failed (%s)' % (name, str(e)))
-            time.sleep(5)
-
-        if not artistResults:
-            return False
-        for result in artistResults:
-            if 'disambiguation' in result:
-                uniquename = unicode(result['sort-name'] + " (" + result['disambiguation'] + ")")
-            else:
-                uniquename = unicode(result['sort-name'])
-            if result['name'] != uniquename and limit == 1:
-                logger.info('Found an artist with a disambiguation: %s - doing an album based search' % name)
-                artistdict = findArtistbyAlbum(name)
-                if not artistdict:
-                    logger.info('Cannot determine the best match from an artist/album search. Using top match instead')
-                    artistlist.append({
-                    # Just need the artist id if the limit is 1
-                    #    'name':             unicode(result['sort-name']),
-                    #    'uniquename':        uniquename,
-                        'id':                unicode(result['id']),
-                    #    'url':                 unicode("http://musicbrainz.org/artist/" + result['id']),#probably needs to be changed
-                    #    'score':            int(result['ext:score'])
-                        })
-                else:
-                    artistlist.append(artistdict)
-            else:
-                artistlist.append({
-                        'name':             unicode(result['sort-name']),
-                        'uniquename':       uniquename,
-                        'id':               unicode(result['id']),
-                        'url':              unicode("http://musicbrainz.org/artist/" + result['id']),#probably needs to be changed
-                        'score':            int(result['ext:score'])
-                        })
-        return artistlist
+     if not artistResults:
+         return False
+     for result in artistResults:
+         if 'disambiguation' in result:
+             uniquename = unicode(result['sort-name'] + " (" + result['disambiguation'] + ")")
+         else:
+             uniquename = unicode(result['sort-name'])
+         if result['name'] != uniquename and limit == 1:
+             logger.info('Found an artist with a disambiguation: %s - doing an album based search' % name)
+             artistdict = findArtistbyAlbum(name)
+             if not artistdict:
+                 logger.info('Cannot determine the best match from an artist/album search. Using top match instead')
+                 artistlist.append({
+                 # Just need the artist id if the limit is 1
+                 #    'name':             unicode(result['sort-name']),
+                 #    'uniquename':        uniquename,
+                     'id': unicode(result['id']),
+                 #    'url':                 unicode("http://musicbrainz.org/artist/" + result['id']),#probably needs to be changed
+                 #    'score':            int(result['ext:score'])
+                     })
+             else:
+                 artistlist.append(artistdict)
+         else:
+             artistlist.append({
+                     'name': unicode(result['sort-name']),
+                     'uniquename': uniquename,
+                     'id': unicode(result['id']),
+                     'url': unicode("http://musicbrainz.org/artist/" + result['id']),#probably needs to be changed
+                     'score': int(result['ext:score'])
+                     })
+     return artistlist

 def findRelease(name, limit=1, artist=None):
+    releaselist = []
+    releaseResults = None
+
+    # additional artist search
+    if not artist and ':' in name:
+        name, artist = name.rsplit(":", 1)
+
+    chars = set('!?*-')
+    if any((c in chars) for c in name):
+        name = '"' + name + '"'
+    if artist and any((c in chars) for c in artist):
+        artist = '"' + artist + '"'

    with mb_lock:
-        releaselist = []
-        releaseResults = None
-
-        # additional artist search
-        if not artist and ':' in name:
-            name, artist = name.rsplit(":",1)
-
-        chars = set('!?*-')
-        if any((c in chars) for c in name):
-            name = '"'+name+'"'
-        if artist and any((c in chars) for c in artist):
-            artist = '"'+artist+'"'
-
        try:
-            releaseResults = musicbrainzngs.search_releases(query=name,limit=limit,artist=artist)['release-list']
+            releaseResults = musicbrainzngs.search_releases(query=name, limit=limit, artist=artist)['release-list']
        except musicbrainzngs.WebServiceError as e: #need to update exceptions
            logger.warn('Attempt to query MusicBrainz for "%s" failed: %s' % (name, str(e)))
-            time.sleep(5)
+            mb_lock.snooze(5)

-        if not releaseResults:
-            return False
+    if not releaseResults:
+        return False

-        for result in releaseResults:
+    for result in releaseResults:

-            title = result['title']
-            if 'disambiguation' in result:
-                title += ' (' + result['disambiguation'] + ')'
+        title = result['title']
+        if 'disambiguation' in result:
+            title += ' (' + result['disambiguation'] + ')'

-            # Get formats and track counts
-            format_dict = OrderedDict()
-            formats = ''
-            tracks = ''
-            if 'medium-list' in result:
-                for medium in result['medium-list']:
-                    if 'format' in medium:
-                        format = medium['format']
-                        if format not in format_dict:
-                            format_dict[format] = 0
-                        format_dict[format] += 1
-                    if 'track-count' in medium:
-                        if tracks:
-                            tracks += ' + '
-                        tracks += str(medium['track-count'])
-                for format, count in format_dict.items():
-                    if formats:
-                        formats += ' + '
-                    if count > 1:
-                        formats += str(count) + 'x'
-                    formats += format
+        # Get formats and track counts
+        format_dict = OrderedDict()
+        formats = ''
+        tracks = ''
+        if 'medium-list' in result:
+            for medium in result['medium-list']:
+                if 'format' in medium:
+                    format = medium['format']
+                    if format not in format_dict:
+                        format_dict[format] = 0
+                    format_dict[format] += 1
+                if 'track-count' in medium:
+                    if tracks:
+                        tracks += ' + '
+                    tracks += str(medium['track-count'])
+            for format, count in format_dict.items():
+                if formats:
+                    formats += ' + '
+                if count > 1:
+                    formats += str(count) + 'x'
+                formats += format

-            rg_type = ''
-            if 'type' in result['release-group']:
-                rg_type = result['release-group']['type']
-                if rg_type == 'Album' and 'secondary-type-list' in result['release-group']:
-                    secondary_type = result['release-group']['secondary-type-list'][0]
+        rg_type = ''
+        if 'type' in result['release-group']:
+            rg_type = result['release-group']['type']
+            if rg_type == 'Album' and 'secondary-type-list' in result['release-group']:
+                secondary_type = result['release-group']['secondary-type-list'][0]
+                if secondary_type != rg_type:
+                    rg_type = secondary_type
+
+        releaselist.append({
+                    'uniquename': unicode(result['artist-credit'][0]['artist']['name']),
+                    'title': unicode(title),
+                    'id': unicode(result['artist-credit'][0]['artist']['id']),
+                    'albumid': unicode(result['id']),
+                    'url': unicode("http://musicbrainz.org/artist/" + result['artist-credit'][0]['artist']['id']),#probably needs to be changed
+                    'albumurl': unicode("http://musicbrainz.org/release/" + result['id']),#probably needs to be changed
+                    'score': int(result['ext:score']),
+                    'date': unicode(result['date']) if 'date' in result else '',
+                    'country': unicode(result['country']) if 'country' in result else '',
+                    'formats': unicode(formats),
+                    'tracks': unicode(tracks),
+                    'rgid': unicode(result['release-group']['id']),
+                    'rgtype': unicode(rg_type)
+                    })
+    return releaselist
+
+def findSeries(name, limit=1):
+     serieslist = []
+     seriesResults = None
+
+     chars = set('!?*-')
+     if any((c in chars) for c in name):
+         name = '"' + name + '"'
+
+     criteria = {'series': name.lower()}
+
+     with mb_lock:
+         try:
+             seriesResults = musicbrainzngs.search_series(limit=limit, **criteria)['series-list']
+         except musicbrainzngs.WebServiceError as e:
+             logger.warn('Attempt to query MusicBrainz for %s failed (%s)' % (name, str(e)))
+             mb_lock.snooze(5)
+
+     if not seriesResults:
+         return False
+     for result in seriesResults:
+         if 'disambiguation' in result:
+             uniquename = unicode(result['name'] + " (" + result['disambiguation'] + ")")
+         else:
+             uniquename = unicode(result['name'])
+         serieslist.append({
+                     'uniquename': uniquename,
+                     'name': unicode(result['name']),
+                     'type': unicode(result['type']),
+                     'id': unicode(result['id']),
+                     'url': unicode("http://musicbrainz.org/series/" + result['id']),#probably needs to be changed
+                     'score': int(result['ext:score'])
+                     })
+     return serieslist
+
+def getArtist(artistid, extrasonly=False):
+    artist_dict = {}
+    artist = None
+    try:
+        limit = 100
+        with mb_lock:
+            artist = musicbrainzngs.get_artist_by_id(artistid)['artist']
+        newRgs = None
+        artist['release-group-list'] = []
+        while newRgs is None or len(newRgs) >= limit:
+            with mb_lock:
+                newRgs = musicbrainzngs.browse_release_groups(
+                    artistid,
+                    release_type="album",
+                    offset=len(artist['release-group-list']),
+                    limit=limit)
+            newRgs = newRgs['release-group-list']
+            artist['release-group-list'] += newRgs
+    except musicbrainzngs.WebServiceError as e:
+        logger.warn('Attempt to retrieve artist information from MusicBrainz failed for artistid: %s (%s)' % (artistid, str(e)))
+        mb_lock.snooze(5)
+    except Exception as e:
+        pass
+
+    if not artist:
+        return False
+
+    artist_dict['artist_name'] = unicode(artist['name'])
+
+    releasegroups = []
+
+    if not extrasonly:
+        for rg in artist['release-group-list']:
+            if "secondary-type-list" in rg.keys(): #only add releases without a secondary type
+                continue
+            releasegroups.append({
+                'title': unicode(rg['title']),
+                'id': unicode(rg['id']),
+                'url': u"http://musicbrainz.org/release-group/" + rg['id'],
+                'type': unicode(rg['type'])
+            })
+
+    # See if we need to grab extras. Artist specific extras take precedence over global option
+    # Global options are set when adding a new artist
+    myDB = db.DBConnection()
+
+    try:
+        db_artist = myDB.action('SELECT IncludeExtras, Extras from artists WHERE ArtistID=?', [artistid]).fetchone()
+        includeExtras = db_artist['IncludeExtras']
+    except IndexError:
+        includeExtras = False
+
+    if includeExtras:
+
+        # Need to convert extras string from something like '2,5.6' to ['ep','live','remix'] (append new extras to end)
+        if db_artist['Extras']:
+            extras = map(int, db_artist['Extras'].split(','))
+        else:
+            extras = []
+        extras_list = headphones.POSSIBLE_EXTRAS
+
+        includes = []
+
+        i = 1
+        for extra in extras_list:
+            if i in extras:
+                includes.append(extra)
+            i += 1
+
+        for include in includes:
+
+            mb_extras_list = []
+
+            try:
+                limit = 100
+                newRgs = None
+                while newRgs is None or len(newRgs) >= limit:
+                    with mb_lock:
+                        newRgs = musicbrainzngs.browse_release_groups(
+                            artistid, release_type=include, offset=len(mb_extras_list), limit=limit)
+                    newRgs = newRgs['release-group-list']
+                    mb_extras_list += newRgs
+            except musicbrainzngs.WebServiceError as e:
+                logger.warn('Attempt to retrieve artist information from MusicBrainz failed for artistid: %s (%s)' % (artistid, str(e)))
+                mb_lock.snooze(5)
+
+            for rg in mb_extras_list:
+                rg_type = rg['type']
+                if rg_type == 'Album' and 'secondary-type-list' in rg:
+                    secondary_type = rg['secondary-type-list'][0]
                    if secondary_type != rg_type:
                        rg_type = secondary_type

-            releaselist.append({
-                        'uniquename':        unicode(result['artist-credit'][0]['artist']['name']),
-                        'title':             unicode(title),
-                        'id':                unicode(result['artist-credit'][0]['artist']['id']),
-                        'albumid':           unicode(result['id']),
-                        'url':               unicode("http://musicbrainz.org/artist/" + result['artist-credit'][0]['artist']['id']),#probably needs to be changed
-                        'albumurl':          unicode("http://musicbrainz.org/release/" + result['id']),#probably needs to be changed
-                        'score':             int(result['ext:score']),
-                        'date':              unicode(result['date']) if 'date' in result else '',
-                        'country':           unicode(result['country']) if 'country' in result else '',
-                        'formats':           unicode(formats),
-                        'tracks':            unicode(tracks),
-                        'rgid':              unicode(result['release-group']['id']),
-                        'rgtype':            unicode(rg_type)
-                        })
-        return releaselist
-
-def getArtist(artistid, extrasonly=False):
-
-    with mb_lock:
-        artist_dict = {}
-
-        artist = None
-
-        try:
-            limit = 200
-            artist = musicbrainzngs.get_artist_by_id(artistid)['artist']
-            newRgs = None
-            artist['release-group-list'] = []
-            while newRgs == None or len(newRgs) >= limit:
-                newRgs = musicbrainzngs.browse_release_groups(artistid,release_type="album",offset=len(artist['release-group-list']),limit=limit)['release-group-list']
-                artist['release-group-list'] += newRgs
-        except musicbrainzngs.WebServiceError as e:
-            logger.warn('Attempt to retrieve artist information from MusicBrainz failed for artistid: %s (%s)' % (artistid, str(e)))
-            time.sleep(5)
-        except Exception,e:
-            pass
-
-        if not artist:
-            return False
-
-        #if 'disambiguation' in artist:
-        #    uniquename = unicode(artist['sort-name'] + " (" + artist['disambiguation'] + ")")
-        #else:
-        #    uniquename = unicode(artist['sort-name'])
-
-        artist_dict['artist_name'] = unicode(artist['name'])
-
-        # Not using the following values anywhere yet so we don't need to grab them.
-        # Was causing an exception to be raised if they didn't exist.
-        #
-        #artist_dict['artist_sortname'] = unicode(artist['sort-name'])
-        #artist_dict['artist_uniquename'] = uniquename
-        #artist_dict['artist_type'] = unicode(artist['type'])
-
-        #artist_dict['artist_begindate'] = None
-        #artist_dict['artist_enddate'] = None
-        #if 'life-span' in artist:
-        #    if 'begin' in artist['life-span']:
-        #        artist_dict['artist_begindate'] = unicode(artist['life-span']['begin'])
-        #    if 'end' in artist['life-span']:
-        #        artist_dict['artist_enddate'] = unicode(artist['life-span']['end'])
-
-
-        releasegroups = []
-
-        if not extrasonly:
-            for rg in artist['release-group-list']:
-                if "secondary-type-list" in rg.keys(): #only add releases without a secondary type
-                    continue
                releasegroups.append({
-                            'title':      unicode(rg['title']),
-                            'id':         unicode(rg['id']),
-                            'url':        u"http://musicbrainz.org/release-group/" + rg['id'],
-                            'type':       unicode(rg['type'])
-                    })
+                    'title': unicode(rg['title']),
+                    'id': unicode(rg['id']),
+                    'url': u"http://musicbrainz.org/release-group/" + rg['id'],
+                    'type': unicode(rg_type)
+                })
+    artist_dict['releasegroups'] = releasegroups
+    return artist_dict

-        # See if we need to grab extras. Artist specific extras take precedence over global option
-        # Global options are set when adding a new artist
-        myDB = db.DBConnection()
+def getSeries(seriesid):
+    series_dict = {}
+    series = None
+    try:
+        with mb_lock:
+            series = musicbrainzngs.get_series_by_id(seriesid,includes=['release-group-rels'])['series']
+    except musicbrainzngs.WebServiceError as e:
+        logger.warn('Attempt to retrieve series information from MusicBrainz failed for seriesid: %s (%s)' % (seriesid, str(e)))
+        mb_lock.snooze(5)
+    except Exception as e:
+        pass

-        try:
-            db_artist = myDB.action('SELECT IncludeExtras, Extras from artists WHERE ArtistID=?', [artistid]).fetchone()
-            includeExtras = db_artist['IncludeExtras']
-        except IndexError:
-            includeExtras = False
+    if not series:
+        return False
+    
+    if 'disambiguation' in series:
+        series_dict['artist_name'] = unicode(series['name'] + " (" + unicode(series['disambiguation']) + ")")
+    else:
+        series_dict['artist_name'] = unicode(series['name'])

-        if includeExtras:
+    releasegroups = []

-            # Need to convert extras string from something like '2,5.6' to ['ep','live','remix'] (append new extras to end)
-            if db_artist['Extras']:
-                extras = map(int, db_artist['Extras'].split(','))
-            else:
-                extras = []
-            extras_list = ["single", "ep", "compilation", "soundtrack", "live", "remix", "spokenword", "audiobook", "other", "dj-mix", "mixtape/street", "broadcast", "interview", "demo"]
-
-            includes = []
-
-            i = 1
-            for extra in extras_list:
-                if i in extras:
-                    includes.append(extra)
-                i += 1
-
-            for include in includes:
-
-                mb_extras_list = []
-
-                try:
-                    limit = 200
-                    newRgs = None
-                    while newRgs == None or len(newRgs) >= limit:
-                        newRgs = musicbrainzngs.browse_release_groups(artistid,release_type=include,offset=len(mb_extras_list),limit=limit)['release-group-list']
-                        mb_extras_list += newRgs
-                except musicbrainzngs.WebServiceError as e:
-                    logger.warn('Attempt to retrieve artist information from MusicBrainz failed for artistid: %s (%s)' % (artistid, str(e)))
-                    time.sleep(5)
-
-                for rg in mb_extras_list:
-
-                    rg_type = rg['type']
-                    if rg_type == 'Album' and 'secondary-type-list' in rg:
-                        secondary_type = rg['secondary-type-list'][0]
-                        if secondary_type != rg_type:
-                            rg_type = secondary_type
-
-                    releasegroups.append({
-                            'title':        unicode(rg['title']),
-                            'id':           unicode(rg['id']),
-                            'url':          u"http://musicbrainz.org/release-group/" + rg['id'],
-                            'type':         unicode(rg_type)
-                        })
-
-        artist_dict['releasegroups'] = releasegroups
-
-        return artist_dict
+    for rg in series['release_group-relation-list']:
+        releasegroup = rg['release-group']
+        releasegroups.append({
+            'title':releasegroup['title'],
+            'date':releasegroup['first-release-date'],
+            'id':releasegroup['id'],
+            'type':rg['type']
+            })
+    series_dict['releasegroups'] = releasegroups
+    return series_dict

 def getReleaseGroup(rgid):
    """
    Returns a list of releases in a release group
    """
-    with mb_lock:
+    releaseGroup = None
+    try:
+        with mb_lock:
+            releaseGroup = musicbrainzngs.get_release_group_by_id(
+                rgid, ["artists", "releases", "media", "discids", ])
+            releaseGroup = releaseGroup['release-group']
+    except musicbrainzngs.WebServiceError as e:
+        logger.warn('Attempt to retrieve information from MusicBrainz for release group "%s" failed (%s)' % (rgid, str(e)))
+        mb_lock.snooze(5)

-        releaselist = []
+    if not releaseGroup:
+        return False
+    else:
+        return releaseGroup['release-list']

-        releaseGroup = None
-
-        try:
-            releaseGroup = musicbrainzngs.get_release_group_by_id(rgid,["artists","releases","media","discids",])['release-group']
-        except musicbrainzngs.WebServiceError as e:
-            logger.warn('Attempt to retrieve information from MusicBrainz for release group "%s" failed (%s)' % (rgid, str(e)))
-            time.sleep(5)
-
-        if not releaseGroup:
-            return False
-        else:
-            return releaseGroup['release-list']

 def getRelease(releaseid, include_artist_info=True):
    """
    Deep release search to get track info
    """
-    with mb_lock:
+    release = {}
+    results = None

-        release = {}
-        results = None
-
-        try:
+    try:
+        with mb_lock:
            if include_artist_info:
-                results = musicbrainzngs.get_release_by_id(releaseid,["artists","release-groups","media","recordings"]).get('release')
+                results = musicbrainzngs.get_release_by_id(releaseid, ["artists", "release-groups", "media", "recordings"]).get('release')
            else:
-                results = musicbrainzngs.get_release_by_id(releaseid,["media","recordings"]).get('release')
-        except musicbrainzngs.WebServiceError as e:
-            logger.warn('Attempt to retrieve information from MusicBrainz for release "%s" failed (%s)' % (releaseid, str(e)))
-            time.sleep(5)
+                results = musicbrainzngs.get_release_by_id(releaseid, ["media", "recordings"]).get('release')
+    except musicbrainzngs.WebServiceError as e:
+        logger.warn('Attempt to retrieve information from MusicBrainz for release "%s" failed (%s)' % (releaseid, str(e)))
+        mb_lock.snooze(5)

-        if not results:
-            return False
+    if not results:
+        return False

-        release['title'] = unicode(results['title'])
-        release['id'] = unicode(results['id'])
-        release['asin'] = unicode(results['asin']) if 'asin' in results else None
-        release['date'] = unicode(results['date']) if 'date' in results else None
-        try:
-            release['format'] = unicode(results['medium-list'][0]['format'])
-        except:
-            release['format'] = u'Unknown'
+    release['title'] = unicode(results['title'])
+    release['id'] = unicode(results['id'])
+    release['asin'] = unicode(results['asin']) if 'asin' in results else None
+    release['date'] = unicode(results['date']) if 'date' in results else None
+    try:
+        release['format'] = unicode(results['medium-list'][0]['format'])
+    except:
+        release['format'] = u'Unknown'

-        try:
-            release['country'] = unicode(results['country'])
-        except:
-            release['country'] = u'Unknown'
+    try:
+        release['country'] = unicode(results['country'])
+    except:
+        release['country'] = u'Unknown'
+
+    if include_artist_info:
+
+        if 'release-group' in results:
+            release['rgid'] = unicode(results['release-group']['id'])
+            release['rg_title'] = unicode(results['release-group']['title'])
+            try:
+                release['rg_type'] = unicode(results['release-group']['type'])
+
+                if release['rg_type'] == 'Album' and 'secondary-type-list' in results['release-group']:
+                    secondary_type = unicode(results['release-group']['secondary-type-list'][0])
+                    if secondary_type != release['rg_type']:
+                        release['rg_type'] = secondary_type
+
+            except KeyError:
+                release['rg_type'] = u'Unknown'
+
+        else:
+            logger.warn("Release " + releaseid + "had no ReleaseGroup associated")
+
+        release['artist_name'] = unicode(results['artist-credit'][0]['artist']['name'])
+        release['artist_id'] = unicode(results['artist-credit'][0]['artist']['id'])
+
+    release['tracks'] = getTracksFromRelease(results)
+
+    return release


-        if include_artist_info:
-
-            if 'release-group' in results:
-                release['rgid'] = unicode(results['release-group']['id'])
-                release['rg_title'] = unicode(results['release-group']['title'])
-                try:
-                    release['rg_type'] = unicode(results['release-group']['type'])
-
-                    if release['rg_type'] == 'Album' and 'secondary-type-list' in results['release-group']:
-                        secondary_type = unicode(results['release-group']['secondary-type-list'][0])
-                        if secondary_type != release['rg_type']:
-                            release['rg_type'] = secondary_type
-
-                except KeyError:
-                    release['rg_type'] = u'Unknown'
-
-            else:
-                logger.warn("Release " + releaseid + "had no ReleaseGroup associated")
-
-            release['artist_name'] = unicode(results['artist-credit'][0]['artist']['name'])
-            release['artist_id'] = unicode(results['artist-credit'][0]['artist']['id'])
-
-        release['tracks'] = getTracksFromRelease(results)
-
-        return release
-
-def get_new_releases(rgid,includeExtras=False,forcefull=False):
+def get_new_releases(rgid, includeExtras=False, forcefull=False):

    myDB = db.DBConnection()
    results = []
    try:
        limit = 100
        newResults = None
-        while newResults == None or len(newResults) >= limit:
-            newResults = musicbrainzngs.browse_releases(release_group=rgid,includes=['artist-credits','labels','recordings','release-groups','media'],limit=limit,offset=len(results))
+        while newResults is None or len(newResults) >= limit:
+            with mb_lock:
+                newResults = musicbrainzngs.browse_releases(
+                    release_group=rgid,
+                    includes=['artist-credits', 'labels', 'recordings', 'release-groups', 'media'],
+                    limit=limit,
+                    offset=len(results))
            if 'release-list' not in newResults:
                break #may want to raise an exception here instead ?
            newResults = newResults['release-list']
@@ -420,7 +483,7 @@ def get_new_releases(rgid,includeExtras=False,forcefull=False):

    except musicbrainzngs.WebServiceError as e:
        logger.warn('Attempt to retrieve information from MusicBrainz for release group "%s" failed (%s)' % (rgid, str(e)))
-        time.sleep(5)
+        mb_lock.snooze(5)
        return False

    if not results or len(results) == 0:
@@ -457,8 +520,6 @@ def get_new_releases(rgid,includeExtras=False,forcefull=False):

        release = {}
        rel_id_check = releasedata['id']
-        artistid = unicode(releasedata['artist-credit'][0]['artist']['id'])
-
        album_checker = myDB.action('SELECT * from allalbums WHERE ReleaseID=?', [rel_id_check]).fetchone()
        if not album_checker or forcefull:
            #DELETE all references to this release since we're updating it anyway.
@@ -486,21 +547,20 @@ def get_new_releases(rgid,includeExtras=False,forcefull=False):
                logger.warn('Release ' + releasedata['id'] + ' has no Artists associated.')
                return False

-
            release['ReleaseCountry'] = unicode(releasedata['country']) if 'country' in releasedata else u'Unknown'
            #assuming that the list will contain media and that the format will be consistent
            try:
-                additional_medium=''
+                additional_medium = ''
                for position in releasedata['medium-list']:
                    if position['format'] == releasedata['medium-list'][0]['format']:
                        medium_count = int(position['position'])
                    else:
-                        additional_medium = additional_medium+' + '+position['format']
+                        additional_medium = additional_medium + ' + ' + position['format']
                if medium_count == 1:
                    disc_number = ''
                else:
-                    disc_number = str(medium_count)+'x'
-                packaged_medium = disc_number+releasedata['medium-list'][0]['format']+additional_medium
+                    disc_number = str(medium_count) + 'x'
+                packaged_medium = disc_number + releasedata['medium-list'][0]['format'] + additional_medium
                release['ReleaseFormat'] = unicode(packaged_medium)
            except:
                release['ReleaseFormat'] = u'Unknown'
@@ -510,17 +570,17 @@ def get_new_releases(rgid,includeExtras=False,forcefull=False):
        # What we're doing here now is first updating the allalbums & alltracks table to the most
        # current info, then moving the appropriate release into the album table and its associated
        # tracks into the tracks table
-            controlValueDict = {"ReleaseID" : release['ReleaseID']}
+            controlValueDict = {"ReleaseID": release['ReleaseID']}

-            newValueDict = {"ArtistID":         release['ArtistID'],
-                            "ArtistName":       release['ArtistName'],
-                            "AlbumTitle":       release['AlbumTitle'],
-                            "AlbumID":          release['AlbumID'],
-                            "AlbumASIN":        release['AlbumASIN'],
-                            "ReleaseDate":      release['ReleaseDate'],
-                            "Type":             release['Type'],
-                            "ReleaseCountry":   release['ReleaseCountry'],
-                            "ReleaseFormat":    release['ReleaseFormat']
+            newValueDict = {"ArtistID": release['ArtistID'],
+                            "ArtistName": release['ArtistName'],
+                            "AlbumTitle": release['AlbumTitle'],
+                            "AlbumID": release['AlbumID'],
+                            "AlbumASIN": release['AlbumASIN'],
+                            "ReleaseDate": release['ReleaseDate'],
+                            "Type": release['Type'],
+                            "ReleaseCountry": release['ReleaseCountry'],
+                            "ReleaseFormat": release['ReleaseFormat']
                        }

            myDB.upsert("allalbums", newValueDict, controlValueDict)
@@ -529,18 +589,18 @@ def get_new_releases(rgid,includeExtras=False,forcefull=False):

                cleanname = helpers.cleanName(release['ArtistName'] + ' ' + release['AlbumTitle'] + ' ' + track['title'])

-                controlValueDict = {"TrackID":      track['id'],
-                                    "ReleaseID":    release['ReleaseID']}
+                controlValueDict = {"TrackID": track['id'],
+                                    "ReleaseID": release['ReleaseID']}

-                newValueDict = {"ArtistID":         release['ArtistID'],
-                                "ArtistName":       release['ArtistName'],
-                                "AlbumTitle":       release['AlbumTitle'],
-                                "AlbumID":          release['AlbumID'],
-                                "AlbumASIN":        release['AlbumASIN'],
-                                "TrackTitle":       track['title'],
-                                "TrackDuration":    track['duration'],
-                                "TrackNumber":      track['number'],
-                                "CleanName":        cleanname
+                newValueDict = {"ArtistID": release['ArtistID'],
+                                "ArtistName": release['ArtistName'],
+                                "AlbumTitle": release['AlbumTitle'],
+                                "AlbumID": release['AlbumID'],
+                                "AlbumASIN": release['AlbumASIN'],
+                                "TrackTitle": track['title'],
+                                "TrackDuration": track['duration'],
+                                "TrackNumber": track['number'],
+                                "CleanName": cleanname
                            }

                match = myDB.action('SELECT Location, BitRate, Format from have WHERE CleanName=?', [cleanname]).fetchone()
@@ -558,20 +618,19 @@ def get_new_releases(rgid,includeExtras=False,forcefull=False):

                myDB.upsert("alltracks", newValueDict, controlValueDict)
            num_new_releases = num_new_releases + 1
-            #print releasedata['title']
-            #print num_new_releases
            if album_checker:
                logger.info('[%s] Existing release %s (%s) updated' % (release['ArtistName'], release['AlbumTitle'], rel_id_check))
            else:
                logger.info('[%s] New release %s (%s) added' % (release['ArtistName'], release['AlbumTitle'], rel_id_check))
-    if force_repackage1 == 1:
-        num_new_releases = -1
-        logger.info('[%s] Forcing repackage of %s, since dB releases have been removed' % (release['ArtistName'], release_title))
-    else:
-        num_new_releases = num_new_releases
+        if force_repackage1 == 1:
+            num_new_releases = -1
+            logger.info('[%s] Forcing repackage of %s, since dB releases have been removed' % (release['ArtistName'], release_title))
+        else:
+            num_new_releases = num_new_releases

    return num_new_releases

+
 def getTracksFromRelease(release):
    totalTracks = 1
    tracks = []
@@ -582,16 +641,18 @@ def getTracksFromRelease(release):
            except:
                track_title = unicode(track['recording']['title'])
            tracks.append({
-                    'number':        totalTracks,
-                    'title':         track_title,
-                    'id':            unicode(track['recording']['id']),
-                    'url':           u"http://musicbrainz.org/track/" + track['recording']['id'],
-                    'duration':      int(track['length']) if 'length' in track else 0
+                    'number': totalTracks,
+                    'title': track_title,
+                    'id': unicode(track['recording']['id']),
+                    'url': u"http://musicbrainz.org/track/" + track['recording']['id'],
+                    'duration': int(track['length']) if 'length' in track else 0
                    })
            totalTracks += 1
    return tracks

 # Used when there is a disambiguation
+
+
 def findArtistbyAlbum(name):

    myDB = db.DBConnection()
@@ -605,16 +666,16 @@ def findArtistbyAlbum(name):
    if not artist['AlbumTitle']:
        return False

-    term = '"'+artist['AlbumTitle']+'" AND artist:"'+name+'"'
+    term = '"' + artist['AlbumTitle'] + '" AND artist:"' + name + '"'

    results = None

    try:
-        results = musicbrainzngs.search_release_groups(term).get('release-group-list')
+        with mb_lock:
+            results = musicbrainzngs.search_release_groups(term).get('release-group-list')
    except musicbrainzngs.WebServiceError as e:
        logger.warn('Attempt to query MusicBrainz for %s failed (%s)' % (name, str(e)))
-        time.sleep(5)
-
+        mb_lock.snooze(5)

    if not results:
        return False
@@ -633,10 +694,9 @@ def findArtistbyAlbum(name):
        #artist_dict['url'] = u'http://musicbrainz.org/artist/' + newArtist['id']
        #artist_dict['score'] = int(releaseGroup['ext:score'])

-
-
    return artist_dict

+
 def findAlbumID(artist=None, album=None):

    results = None
@@ -645,20 +705,20 @@ def findAlbumID(artist=None, album=None):
    try:
        if album and artist:
            if any((c in chars) for c in album):
-                album = '"'+album+'"'
+                album = '"' + album + '"'
            if any((c in chars) for c in artist):
-                artist = '"'+artist+'"'
+                artist = '"' + artist + '"'
            criteria = {'release': album.lower()}
            criteria['artist'] = artist.lower()
        else:
            if any((c in chars) for c in album):
-                album = '"'+album+'"'
+                album = '"' + album + '"'
            criteria = {'release': album.lower()}
-
-        results = musicbrainzngs.search_release_groups(limit=1, **criteria).get('release-group-list')
+        with mb_lock:
+            results = musicbrainzngs.search_release_groups(limit=1, **criteria).get('release-group-list')
    except musicbrainzngs.WebServiceError as e:
        logger.warn('Attempt to query MusicBrainz for %s - %s failed (%s)' % (artist, album, str(e)))
-        time.sleep(5)
+        mb_lock.snooze(5)

    if not results:
        return False
@@ -0,0 +1,87 @@
+#  This file is part of Headphones.
+#
+#  Headphones is free software: you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation, either version 3 of the License, or
+#  (at your option) any later version.
+#
+#  Headphones is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.
+
+import re
+import json
+import headphones
+
+from headphones import db, helpers, logger, request
+from headphones.common import USER_AGENT
+
+def update(artistid, artist_name,release_groups):
+    """ Pretty simple and crude function to find the artist page on metacritic,
+    then parse that page to get critic & user scores for albums"""
+
+    # First let's modify the artist name to fit the metacritic convention.
+    # We could just do a search, then take the top result, but at least this will
+    # cut down on api calls. If it's ineffective then we'll switch to search
+
+    replacements = {" & " : " ", "." : ""}
+    mc_artist_name = helpers.replace_all(artist_name.lower(),replacements)
+
+    mc_artist_name = mc_artist_name.replace(" ","-")
+
+    headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2243.2 Safari/537.36'}
+
+    url = "http://www.metacritic.com/person/" + mc_artist_name + "?filter-options=music&sort_options=date&num_items=100"
+
+    res = request.request_soup(url, headers=headers, parser='html.parser')
+
+    rows = None
+
+    try:
+        rows = res.tbody.find_all('tr')
+    except:
+        logger.info("Unable to get metacritic scores for: %s" % artist_name)
+
+    myDB = db.DBConnection()
+    artist = myDB.action('SELECT * FROM artists WHERE ArtistID=?', [artistid]).fetchone()
+
+    score_list = []
+
+    # If we couldn't get anything from MetaCritic for whatever reason,
+    # let's try to load scores from the db
+    if not rows:
+        if artist['MetaCritic']:
+            score_list = json.loads(artist['MetaCritic'])
+        else:
+            return
+
+    # If we did get scores, let's update the db with them
+    else:
+        for row in rows:
+            title = row.a.string
+            scores = row.find_all("span")
+            critic_score = scores[0].string
+            user_score = scores[1].string
+            score_dict = {'title':title,'critic_score':critic_score,'user_score':user_score}
+            score_list.append(score_dict)
+
+        # Save scores to the database
+        controlValueDict = {"ArtistID": artistid}
+        newValueDict = {'MetaCritic':json.dumps(score_list)}
+        myDB.upsert("artists", newValueDict, controlValueDict)
+
+    for score in score_list:
+        title = score['title']
+        # Iterate through the release groups we got passed to see if we can find
+        # a match
+        for rg in release_groups:
+            if rg['title'].lower() == title.lower():
+                critic_score = score['critic_score']
+                user_score = score['user_score']
+                controlValueDict = {"AlbumID": rg['id']}
+                newValueDict = {'CriticScore':critic_score,'UserScore':user_score}
+                myDB.upsert("albums", newValueDict, controlValueDict)
@@ -14,36 +14,35 @@
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

 import os
-import headphones
-import shutil
 import time
+import shutil
+import subprocess
+import headphones
 import multiprocessing

-import subprocess
 from headphones import logger
 from beets.mediafile import MediaFile

 # xld
-if headphones.ENCODER == 'xld':
-    import getXldProfile
-    XLD = True
-else:
-    XLD = False
+import getXldProfile
+

 def encode(albumPath):
+    use_xld = headphones.CONFIG.ENCODER == 'xld'

    # Return if xld details not found
-    if XLD:
-        global xldProfile
-        (xldProfile, xldFormat, xldBitrate) = getXldProfile.getXldProfile(headphones.XLDPROFILE)
+    if use_xld:
+        (xldProfile, xldFormat, xldBitrate) = getXldProfile.getXldProfile(headphones.CONFIG.XLDPROFILE)
        if not xldFormat:
            logger.error('Details for xld profile \'%s\' not found, files will not be re-encoded', xldProfile)
            return None
+    else:
+        xldProfile = None

-    tempDirEncode=os.path.join(albumPath,"temp")
-    musicFiles=[]
-    musicFinalFiles=[]
-    musicTempFiles=[]
+    tempDirEncode = os.path.join(albumPath, "temp")
+    musicFiles = []
+    musicFinalFiles = []
+    musicTempFiles = []
    encoder = ""

    # Create temporary directory, but remove the old one first.
@@ -57,19 +56,19 @@ def encode(albumPath):
        logger.exception("Unable to create temporary directory")
        return None

-    for r,d,f in os.walk(albumPath):
+    for r, d, f in os.walk(albumPath):
        for music in f:
            if any(music.lower().endswith('.' + x.lower()) for x in headphones.MEDIA_FORMATS):
-                if not XLD:
-                    encoderFormat = headphones.ENCODEROUTPUTFORMAT.encode(headphones.SYS_ENCODING)
+                if not use_xld:
+                    encoderFormat = headphones.CONFIG.ENCODEROUTPUTFORMAT.encode(headphones.SYS_ENCODING)
                else:
                    xldMusicFile = os.path.join(r, music)
                    xldInfoMusic = MediaFile(xldMusicFile)
                    encoderFormat = xldFormat

-                if (headphones.ENCODERLOSSLESS):
+                if headphones.CONFIG.ENCODERLOSSLESS:
                    ext = os.path.normpath(os.path.splitext(music)[1].lstrip(".")).lower()
-                    if not XLD and ext == 'flac' or XLD and (ext != xldFormat and (xldInfoMusic.bitrate / 1000 > 400)):
+                    if not use_xld and ext == 'flac' or use_xld and (ext != xldFormat and (xldInfoMusic.bitrate / 1000 > 400)):
                        musicFiles.append(os.path.join(r, music))
                        musicTemp = os.path.normpath(os.path.splitext(music)[0] + '.' + encoderFormat)
                        musicTempFiles.append(os.path.join(tempDirEncode, musicTemp))
@@ -80,29 +79,29 @@ def encode(albumPath):
                    musicTemp = os.path.normpath(os.path.splitext(music)[0] + '.' + encoderFormat)
                    musicTempFiles.append(os.path.join(tempDirEncode, musicTemp))

-    if headphones.ENCODER_PATH:
-        encoder = headphones.ENCODER_PATH.encode(headphones.SYS_ENCODING)
+    if headphones.CONFIG.ENCODER_PATH:
+        encoder = headphones.CONFIG.ENCODER_PATH.encode(headphones.SYS_ENCODING)
    else:
-        if XLD:
+        if use_xld:
            encoder = os.path.join('/Applications', 'xld')
-        elif headphones.ENCODER =='lame':
+        elif headphones.CONFIG.ENCODER == 'lame':
            if headphones.SYS_PLATFORM == "win32":
                ## NEED THE DEFAULT LAME INSTALL ON WIN!
                encoder = "C:/Program Files/lame/lame.exe"
            else:
-                encoder="lame"
-        elif headphones.ENCODER =='ffmpeg':
+                encoder = "lame"
+        elif headphones.CONFIG.ENCODER == 'ffmpeg':
            if headphones.SYS_PLATFORM == "win32":
                encoder = "C:/Program Files/ffmpeg/bin/ffmpeg.exe"
            else:
-                encoder="ffmpeg"
-        elif headphones.ENCODER == 'libav':
+                encoder = "ffmpeg"
+        elif headphones.CONFIG.ENCODER == 'libav':
            if headphones.SYS_PLATFORM == "win32":
                encoder = "C:/Program Files/libav/bin/avconv.exe"
            else:
-                encoder="avconv"
+                encoder = "avconv"

-    i=0
+    i = 0
    encoder_failed = False
    jobs = []

@@ -110,50 +109,50 @@ def encode(albumPath):
        infoMusic = MediaFile(music)
        encode = False

-        if XLD:
+        if use_xld:
            if xldBitrate and (infoMusic.bitrate / 1000 <= xldBitrate):
                logger.info('%s has bitrate <= %skb, will not be re-encoded', music.decode(headphones.SYS_ENCODING, 'replace'), xldBitrate)
            else:
                encode = True
-        elif headphones.ENCODER == 'lame':
+        elif headphones.CONFIG.ENCODER == 'lame':
            if not any(music.decode(headphones.SYS_ENCODING, 'replace').lower().endswith('.' + x) for x in ["mp3", "wav"]):
                logger.warn('Lame cannot encode %s format for %s, use ffmpeg', os.path.splitext(music)[1], music)
            else:
-                if (music.decode(headphones.SYS_ENCODING, 'replace').lower().endswith('.mp3') and (int(infoMusic.bitrate / 1000) <= headphones.BITRATE)):
-                    logger.info('%s has bitrate <= %skb, will not be re-encoded', music, headphones.BITRATE)
+                if music.decode(headphones.SYS_ENCODING, 'replace').lower().endswith('.mp3') and (int(infoMusic.bitrate / 1000) <= headphones.CONFIG.BITRATE):
+                    logger.info('%s has bitrate <= %skb, will not be re-encoded', music, headphones.CONFIG.BITRATE)
                else:
                    encode = True
        else:
-            if headphones.ENCODEROUTPUTFORMAT=='ogg':
+            if headphones.CONFIG.ENCODEROUTPUTFORMAT == 'ogg':
                if music.decode(headphones.SYS_ENCODING, 'replace').lower().endswith('.ogg'):
                    logger.warn('Cannot re-encode .ogg %s', music.decode(headphones.SYS_ENCODING, 'replace'))
                else:
                    encode = True
-            elif (headphones.ENCODEROUTPUTFORMAT=='mp3' or headphones.ENCODEROUTPUTFORMAT=='m4a'):
-                if (music.decode(headphones.SYS_ENCODING, 'replace').lower().endswith('.'+headphones.ENCODEROUTPUTFORMAT) and (int(infoMusic.bitrate / 1000 ) <= headphones.BITRATE)):
-                    logger.info('%s has bitrate <= %skb, will not be re-encoded', music, headphones.BITRATE)
+            elif headphones.CONFIG.ENCODEROUTPUTFORMAT == 'mp3' or headphones.CONFIG.ENCODEROUTPUTFORMAT == 'm4a':
+                if music.decode(headphones.SYS_ENCODING, 'replace').lower().endswith('.' + headphones.CONFIG.ENCODEROUTPUTFORMAT) and (int(infoMusic.bitrate / 1000) <= headphones.CONFIG.BITRATE):
+                    logger.info('%s has bitrate <= %skb, will not be re-encoded', music, headphones.CONFIG.BITRATE)
                else:
                    encode = True
        # encode
        if encode:
-            job = (encoder, music, musicTempFiles[i], albumPath)
+            job = (encoder, music, musicTempFiles[i], albumPath, xldProfile)
            jobs.append(job)
        else:
            musicFiles[i] = None
            musicTempFiles[i] = None

-        i=i+1
+        i = i + 1

    # Encode music files
    if len(jobs) > 0:
        processes = 1

        # Use multicore if enabled
-        if headphones.ENCODER_MULTICORE:
-            if headphones.ENCODER_MULTICORE_COUNT == 0:
+        if headphones.CONFIG.ENCODER_MULTICORE:
+            if headphones.CONFIG.ENCODER_MULTICORE_COUNT == 0:
                processes = multiprocessing.cpu_count()
            else:
-                processes = headphones.ENCODER_MULTICORE_COUNT
+                processes = headphones.CONFIG.ENCODER_MULTICORE_COUNT

            logger.debug("Multi-core encoding enabled, spawning %d processes",
                processes)
@@ -194,14 +193,14 @@ def encode(albumPath):
        for dest in musicTempFiles:
            if os.path.exists(dest):
                source = musicFiles[i]
-                if headphones.DELETE_LOSSLESS_FILES:
+                if headphones.CONFIG.DELETE_LOSSLESS_FILES:
                    os.remove(source)
                check_dest = os.path.join(albumPath, os.path.split(dest)[1])
                if os.path.exists(check_dest):
                    os.remove(check_dest)
                try:
                    shutil.move(dest, albumPath)
-                except Exception, e:
+                except Exception as e:
                    logger.error('Could not move %s to %s: %s', dest, albumPath, e)
                    encoder_failed = True
                    break
@@ -212,11 +211,11 @@ def encode(albumPath):

    # Return with error if any encoding errors
    if encoder_failed:
-        logger.error("One or more files failed to encode. Ensure you have the latest version of %s installed.", headphones.ENCODER)
+        logger.error("One or more files failed to encode. Ensure you have the latest version of %s installed.", headphones.CONFIG.ENCODER)
        return None

    time.sleep(1)
-    for r,d,f in os.walk(albumPath):
+    for r, d, f in os.walk(albumPath):
        for music in f:
            if any(music.lower().endswith('.' + x.lower()) for x in headphones.MEDIA_FORMATS):
                musicFinalFiles.append(os.path.join(r, music))
@@ -226,6 +225,7 @@ def encode(albumPath):

    return musicFinalFiles

+
 def command_map(args):
    """
    Wrapper for the '[multiprocessing.]map()' method, to unpack the arguments
@@ -239,11 +239,12 @@ def command_map(args):
    # Start encoding
    try:
        return command(*args)
-    except Exception as e:
+    except Exception:
        logger.exception("Encoder raised an exception.")
        return False

-def command(encoder, musicSource, musicDest, albumPath):
+
+def command(encoder, musicSource, musicDest, albumPath, xldProfile):
    """
    Encode a given music file with a certain encoder. Returns True on success,
    or False otherwise.
@@ -252,8 +253,7 @@ def command(encoder, musicSource, musicDest, albumPath):
    startMusicTime = time.time()
    cmd = []

-    # XLD
-    if XLD:
+    if xldProfile:
        xldDestDir = os.path.split(musicDest)[0]
        cmd = [encoder]
        cmd.extend([musicSource])
@@ -263,17 +263,17 @@ def command(encoder, musicSource, musicDest, albumPath):
        cmd.extend([xldDestDir])

    # Lame
-    elif headphones.ENCODER == 'lame':
+    elif headphones.CONFIG.ENCODER == 'lame':
        cmd = [encoder]
        opts = []
-        if not headphones.ADVANCEDENCODER:
+        if not headphones.CONFIG.ADVANCEDENCODER:
            opts.extend(['-h'])
-            if headphones.ENCODERVBRCBR=='cbr':
-                opts.extend(['--resample', str(headphones.SAMPLINGFREQUENCY), '-b', str(headphones.BITRATE)])
-            elif headphones.ENCODERVBRCBR=='vbr':
-                opts.extend(['-v', str(headphones.ENCODERQUALITY)])
+            if headphones.CONFIG.ENCODERVBRCBR == 'cbr':
+                opts.extend(['--resample', str(headphones.CONFIG.SAMPLINGFREQUENCY), '-b', str(headphones.CONFIG.BITRATE)])
+            elif headphones.CONFIG.ENCODERVBRCBR == 'vbr':
+                opts.extend(['-v', str(headphones.CONFIG.ENCODERQUALITY)])
        else:
-            advanced = (headphones.ADVANCEDENCODER.split())
+            advanced = (headphones.CONFIG.ADVANCEDENCODER.split())
            for tok in advanced:
                opts.extend([tok.encode(headphones.SYS_ENCODING)])
        opts.extend([musicSource])
@@ -281,42 +281,42 @@ def command(encoder, musicSource, musicDest, albumPath):
        cmd.extend(opts)

    # FFmpeg
-    elif headphones.ENCODER == 'ffmpeg':
+    elif headphones.CONFIG.ENCODER == 'ffmpeg':
        cmd = [encoder, '-i', musicSource]
        opts = []
-        if not headphones.ADVANCEDENCODER:
-            if headphones.ENCODEROUTPUTFORMAT=='ogg':
+        if not headphones.CONFIG.ADVANCEDENCODER:
+            if headphones.CONFIG.ENCODEROUTPUTFORMAT == 'ogg':
                opts.extend(['-acodec', 'libvorbis'])
-            if headphones.ENCODEROUTPUTFORMAT=='m4a':
+            if headphones.CONFIG.ENCODEROUTPUTFORMAT == 'm4a':
                opts.extend(['-strict', 'experimental'])
-            if headphones.ENCODERVBRCBR=='cbr':
-                opts.extend(['-ar', str(headphones.SAMPLINGFREQUENCY), '-ab', str(headphones.BITRATE) + 'k'])
-            elif headphones.ENCODERVBRCBR=='vbr':
-                opts.extend(['-aq', str(headphones.ENCODERQUALITY)])
+            if headphones.CONFIG.ENCODERVBRCBR == 'cbr':
+                opts.extend(['-ar', str(headphones.CONFIG.SAMPLINGFREQUENCY), '-ab', str(headphones.CONFIG.BITRATE) + 'k'])
+            elif headphones.CONFIG.ENCODERVBRCBR == 'vbr':
+                opts.extend(['-aq', str(headphones.CONFIG.ENCODERQUALITY)])
            opts.extend(['-y', '-ac', '2', '-vn'])
        else:
-            advanced = (headphones.ADVANCEDENCODER.split())
+            advanced = (headphones.CONFIG.ADVANCEDENCODER.split())
            for tok in advanced:
                opts.extend([tok.encode(headphones.SYS_ENCODING)])
        opts.extend([musicDest])
        cmd.extend(opts)

    # Libav
-    elif headphones.ENCODER == "libav":
+    elif headphones.CONFIG.ENCODER == "libav":
        cmd = [encoder, '-i', musicSource]
        opts = []
-        if not headphones.ADVANCEDENCODER:
-            if headphones.ENCODEROUTPUTFORMAT=='ogg':
+        if not headphones.CONFIG.ADVANCEDENCODER:
+            if headphones.CONFIG.ENCODEROUTPUTFORMAT == 'ogg':
                opts.extend(['-acodec', 'libvorbis'])
-            if headphones.ENCODEROUTPUTFORMAT=='m4a':
+            if headphones.CONFIG.ENCODEROUTPUTFORMAT == 'm4a':
                opts.extend(['-strict', 'experimental'])
-            if headphones.ENCODERVBRCBR=='cbr':
-                opts.extend(['-ar', str(headphones.SAMPLINGFREQUENCY), '-ab', str(headphones.BITRATE) + 'k'])
-            elif headphones.ENCODERVBRCBR=='vbr':
-                opts.extend(['-aq', str(headphones.ENCODERQUALITY)])
+            if headphones.CONFIG.ENCODERVBRCBR == 'cbr':
+                opts.extend(['-ar', str(headphones.CONFIG.SAMPLINGFREQUENCY), '-ab', str(headphones.CONFIG.BITRATE) + 'k'])
+            elif headphones.CONFIG.ENCODERVBRCBR == 'vbr':
+                opts.extend(['-aq', str(headphones.CONFIG.ENCODERQUALITY)])
            opts.extend(['-y', '-ac', '2', '-vn'])
        else:
-            advanced = (headphones.ADVANCEDENCODER.split())
+            advanced = (headphones.CONFIG.ADVANCEDENCODER.split())
            for tok in advanced:
                opts.extend([tok.encode(headphones.SYS_ENCODING)])
        opts.extend([musicDest])
@@ -339,7 +339,7 @@ def command(encoder, musicSource, musicDest, albumPath):
    process = subprocess.Popen(cmd, startupinfo=startupinfo,
        stdin=open(os.devnull, 'rb'), stdout=subprocess.PIPE,
        stderr=subprocess.PIPE)
-    stdout, stderr = process.communicate(headphones.ENCODER)
+    stdout, stderr = process.communicate(headphones.CONFIG.ENCODER)

    # Error if return code not zero
    if process.returncode:
@@ -347,7 +347,7 @@ def command(encoder, musicSource, musicDest, albumPath):
        out = stdout if stdout else stderr
        out = out.decode(headphones.SYS_ENCODING, 'replace')
        outlast2lines = '\n'.join(out.splitlines()[-2:])
-        logger.error('%s error details: %s' % (headphones.ENCODER, outlast2lines))
+        logger.error('%s error details: %s' % (headphones.CONFIG.ENCODER, outlast2lines))
        out = out.rstrip("\n")
        logger.debug(out)
        encoded = False
@@ -357,10 +357,11 @@ def command(encoder, musicSource, musicDest, albumPath):

    return encoded

+
 def getTimeEncode(start):
-    seconds =int(time.time()-start)
+    seconds = int(time.time() - start)
    hours = seconds / 3600
-    seconds -= 3600*hours
+    seconds -= 3600 * hours
    minutes = seconds / 60
-    seconds -= 60*minutes
-    return "%02d:%02d:%02d" % (hours, minutes, seconds)
+    seconds -= 60 * minutes
+    return "%02d:%02d:%02d" % (hours, minutes, seconds)
@@ -17,6 +17,7 @@ from headphones import logger, helpers, common, request

 from xml.dom import minidom
 from httplib import HTTPSConnection
+from urlparse import parse_qsl
 from urllib import urlencode
 from pynma import pynma

@@ -28,26 +29,25 @@ import headphones
 import os.path
 import subprocess
 import gntp.notifier
-import time
 import json

 import oauth2 as oauth
 import pythontwitter as twitter

-try:
-    from urlparse import parse_qsl
-except ImportError:
-    from cgi import parse_qsl
+from email.mime.text import MIMEText
+import smtplib
+import email.utils
+

 class GROWL(object):
    """
-    Growl notifications, for OS X
+    Growl notifications, for OS X.
    """

    def __init__(self):
-        self.enabled = headphones.GROWL_ENABLED
-        self.host = headphones.GROWL_HOST
-        self.password = headphones.GROWL_PASSWORD
+        self.enabled = headphones.CONFIG.GROWL_ENABLED
+        self.host = headphones.CONFIG.GROWL_HOST
+        self.password = headphones.CONFIG.GROWL_PASSWORD

    def conf(self, options):
        return cherrypy.config['config'].get('Growl', options)
@@ -84,14 +84,18 @@ class GROWL(object):
        try:
            growl.register()
        except gntp.notifier.errors.NetworkError:
-            logger.info(u'Growl notification failed: network error')
+            logger.warning(u'Growl notification failed: network error')
            return
        except gntp.notifier.errors.AuthError:
-            logger.info(u'Growl notification failed: authentication error')
+            logger.warning(u'Growl notification failed: authentication error')
            return

+        # Fix message
+        message = message.encode(headphones.SYS_ENCODING, "replace")
+
        # Send it, including an image
-        image_file = os.path.join(str(headphones.PROG_DIR), 'data/images/headphoneslogo.png')
+        image_file = os.path.join(str(headphones.PROG_DIR),
+            "data/images/headphoneslogo.png")

        with open(image_file, 'rb') as f:
            image = f.read()
@@ -104,7 +108,7 @@ class GROWL(object):
                icon=image
            )
        except gntp.notifier.errors.NetworkError:
-            logger.info(u'Growl notification failed: network error')
+            logger.warning(u'Growl notification failed: network error')
            return

        logger.info(u"Growl notifications sent.")
@@ -120,35 +124,36 @@ class GROWL(object):

        self.notify('ZOMG Lazors Pewpewpew!', 'Test Message')

+
 class PROWL(object):
    """
    Prowl notifications.
    """

    def __init__(self):
-        self.enabled = headphones.PROWL_ENABLED
-        self.keys = headphones.PROWL_KEYS
-        self.priority = headphones.PROWL_PRIORITY
+        self.enabled = headphones.CONFIG.PROWL_ENABLED
+        self.keys = headphones.CONFIG.PROWL_KEYS
+        self.priority = headphones.CONFIG.PROWL_PRIORITY

    def conf(self, options):
        return cherrypy.config['config'].get('Prowl', options)

    def notify(self, message, event):
-        if not headphones.PROWL_ENABLED:
+        if not headphones.CONFIG.PROWL_ENABLED:
            return

        http_handler = HTTPSConnection("api.prowlapp.com")

-        data = {'apikey': headphones.PROWL_KEYS,
+        data = {'apikey': headphones.CONFIG.PROWL_KEYS,
                'application': 'Headphones',
                'event': event,
                'description': message.encode("utf-8"),
-                'priority': headphones.PROWL_PRIORITY }
+                'priority': headphones.CONFIG.PROWL_PRIORITY}

        http_handler.request("POST",
                                "/publicapi/add",
-                                headers = {'Content-type': "application/x-www-form-urlencoded"},
-                                body = urlencode(data))
+                                headers={'Content-type': "application/x-www-form-urlencoded"},
+                                body=urlencode(data))
        response = http_handler.getresponse()
        request_status = response.status

@@ -173,6 +178,7 @@ class PROWL(object):

        self.notify('ZOMG Lazors Pewpewpew!', 'Test Message')

+
 class MPC(object):
    """
    MPC library update
@@ -182,8 +188,8 @@ class MPC(object):

        pass

-    def notify( self ):
-        subprocess.call( ["mpc", "update"] )
+    def notify(self):
+        subprocess.call(["mpc", "update"])


 class XBMC(object):
@@ -193,9 +199,9 @@ class XBMC(object):

    def __init__(self):

-        self.hosts = headphones.XBMC_HOST
-        self.username = headphones.XBMC_USERNAME
-        self.password = headphones.XBMC_PASSWORD
+        self.hosts = headphones.CONFIG.XBMC_HOST
+        self.username = headphones.CONFIG.XBMC_USERNAME
+        self.password = headphones.CONFIG.XBMC_PASSWORD

    def _sendhttp(self, host, command):
        url_command = urllib.urlencode(command)
@@ -226,7 +232,7 @@ class XBMC(object):
        hosts = [x.strip() for x in self.hosts.split(',')]

        for host in hosts:
-            logger.info('Sending library update command to XBMC @ '+host)
+            logger.info('Sending library update command to XBMC @ ' + host)
            request = self._sendjson(host, 'AudioLibrary.Scan')

            if not request:
@@ -241,17 +247,17 @@ class XBMC(object):
        time = "3000" # in ms

        for host in hosts:
-            logger.info('Sending notification command to XMBC @ '+host)
+            logger.info('Sending notification command to XMBC @ ' + host)
            try:
                version = self._sendjson(host, 'Application.GetProperties', {'properties': ['version']})['version']['major']

                if version < 12: #Eden
                    notification = header + "," + message + "," + time + "," + albumartpath
-                    notifycommand = {'command': 'ExecBuiltIn', 'parameter': 'Notification('+notification+')'}
+                    notifycommand = {'command': 'ExecBuiltIn', 'parameter': 'Notification(' + notification + ')'}
                    request = self._sendhttp(host, notifycommand)

                else: #Frodo
-                    params = {'title':header, 'message': message, 'displaytime': int(time), 'image': albumartpath}
+                    params = {'title': header, 'message': message, 'displaytime': int(time), 'image': albumartpath}
                    request = self._sendjson(host, 'GUI.ShowNotification', params)

                if not request:
@@ -260,25 +266,26 @@ class XBMC(object):
            except Exception:
                logger.error('Error sending notification request to XBMC')

+
 class LMS(object):
    """
    Class for updating a Logitech Media Server
    """

    def __init__(self):
-        self.hosts = headphones.LMS_HOST
+        self.hosts = headphones.CONFIG.LMS_HOST

    def _sendjson(self, host):
-        data = {'id': 1, 'method': 'slim.request', 'params': ["",["rescan"]]}
+        data = {'id': 1, 'method': 'slim.request', 'params': ["", ["rescan"]]}
        data = json.JSONEncoder().encode(data)

        content = {'Content-Type': 'application/json'}

-        req = urllib2.Request(host+'/jsonrpc.js', data, content)
+        req = urllib2.Request(host + '/jsonrpc.js', data, content)

        try:
            handle = urllib2.urlopen(req)
-        except Exception, e:
+        except Exception as e:
            logger.warn('Error opening LMS url: %s' % e)
            return

@@ -288,26 +295,28 @@ class LMS(object):
            return response['result']
        except:
            logger.warn('LMS returned error: %s' % response['error'])
-            return
+            return response['error']

    def update(self):

        hosts = [x.strip() for x in self.hosts.split(',')]

        for host in hosts:
-            logger.info('Sending library rescan command to LMS @ '+host)
+            logger.info('Sending library rescan command to LMS @ ' + host)
            request = self._sendjson(host)

-            if not request:
+            if request:
                logger.warn('Error sending rescan request to LMS')

+
 class Plex(object):
    def __init__(self):

-        self.server_hosts = headphones.PLEX_SERVER_HOST
-        self.client_hosts = headphones.PLEX_CLIENT_HOST
-        self.username = headphones.PLEX_USERNAME
-        self.password = headphones.PLEX_PASSWORD
+        self.server_hosts = headphones.CONFIG.PLEX_SERVER_HOST
+        self.client_hosts = headphones.CONFIG.PLEX_CLIENT_HOST
+        self.username = headphones.CONFIG.PLEX_USERNAME
+        self.password = headphones.CONFIG.PLEX_PASSWORD
+        self.token = headphones.CONFIG.PLEX_TOKEN

    def _sendhttp(self, host, command):

@@ -328,7 +337,7 @@ class Plex(object):

        try:
            handle = urllib2.urlopen(req)
-        except Exception, e:
+        except Exception as e:
            logger.warn('Error opening Plex url: %s' % e)
            return

@@ -344,15 +353,17 @@ class Plex(object):
        hosts = [x.strip() for x in self.server_hosts.split(',')]

        for host in hosts:
-            logger.info('Sending library update command to Plex Media Server@ '+host)
+            logger.info('Sending library update command to Plex Media Server@ ' + host)
            url = "%s/library/sections" % host
-            try:
-                xml_sections = minidom.parse(urllib.urlopen(url))
-            except IOError, e:
-                logger.warn("Error while trying to contact Plex Media Server: %s" % e)
-                return False
+            if self.token:
+                params = {'X-Plex-Token': self.token}
+            else:
+                params = False
+
+            r = request.request_minidom(url, params=params)
+
+            sections = r.getElementsByTagName('Directory')

-            sections = xml_sections.getElementsByTagName('Directory')
            if not sections:
                logger.info(u"Plex Media Server not running on: " + host)
                return False
@@ -360,11 +371,7 @@ class Plex(object):
            for s in sections:
                if s.getAttribute('type') == "artist":
                    url = "%s/library/sections/%s/refresh" % (host, s.getAttribute('key'))
-                    try:
-                        urllib.urlopen(url)
-                    except Exception, e:
-                        logger.warn("Error updating library section for Plex Media Server: %s" % e)
-                        return False
+                    request.request_response(url, params=params)

    def notify(self, artist, album, albumartpath):

@@ -375,10 +382,10 @@ class Plex(object):
        time = "3000" # in ms

        for host in hosts:
-            logger.info('Sending notification command to Plex Media Server @ '+host)
+            logger.info('Sending notification command to Plex Media Server @ ' + host)
            try:
                notification = header + "," + message + "," + time + "," + albumartpath
-                notifycommand = {'command': 'ExecBuiltIn', 'parameter': 'Notification('+notification+')'}
+                notifycommand = {'command': 'ExecBuiltIn', 'parameter': 'Notification(' + notification + ')'}
                request = self._sendhttp(host, notifycommand)

                if not request:
@@ -387,11 +394,12 @@ class Plex(object):
            except:
                logger.warn('Error sending notification request to Plex Media Server')

+
 class NMA(object):
    def notify(self, artist=None, album=None, snatched=None):
        title = 'Headphones'
-        api = headphones.NMA_APIKEY
-        nma_priority = headphones.NMA_PRIORITY
+        api = headphones.CONFIG.NMA_APIKEY
+        nma_priority = headphones.CONFIG.NMA_PRIORITY

        logger.debug(u"NMA title: " + title)
        logger.debug(u"NMA API: " + api)
@@ -413,7 +421,8 @@ class NMA(object):
        keys = api.split(',')
        p.addkey(keys)

-        if len(keys) > 1: batch = True
+        if len(keys) > 1:
+            batch = True

        response = p.push(title, event, message, priority=nma_priority, batch_mode=batch)

@@ -423,31 +432,31 @@ class NMA(object):
        else:
            return True

+
 class PUSHBULLET(object):

    def __init__(self):
-        self.apikey = headphones.PUSHBULLET_APIKEY
-        self.deviceid = headphones.PUSHBULLET_DEVICEID
+        self.apikey = headphones.CONFIG.PUSHBULLET_APIKEY
+        self.deviceid = headphones.CONFIG.PUSHBULLET_DEVICEID

    def conf(self, options):
        return cherrypy.config['config'].get('PUSHBULLET', options)

    def notify(self, message, event):
-        if not headphones.PUSHBULLET_ENABLED:
+        if not headphones.CONFIG.PUSHBULLET_ENABLED:
            return

        http_handler = HTTPSConnection("api.pushbullet.com")

-        data = {'device_iden': headphones.PUSHBULLET_DEVICEID,
-                'type': "note",
+        data = {'type': "note",
                'title': "Headphones",
-                'body': message.encode("utf-8") }
+                'body': message.encode("utf-8")}

        http_handler.request("POST",
-                                "/api/pushes",
-                                headers = {'Content-type': "application/x-www-form-urlencoded",
-                                            'Authorization' : 'Basic %s' % base64.b64encode(headphones.PUSHBULLET_APIKEY + ":") },
-                                body = urlencode(data))
+                                "/v2/pushes",
+                                headers={'Content-type': "application/json",
+                                            'Authorization': 'Basic %s' % base64.b64encode(headphones.CONFIG.PUSHBULLET_APIKEY + ":")},
+                                body=json.dumps(data))
        response = http_handler.getresponse()
        request_status = response.status
        logger.debug(u"PushBullet response status: %r" % request_status)
@@ -476,13 +485,14 @@ class PUSHBULLET(object):

        self.notify('Main Screen Activate', 'Test Message')

+
 class PUSHALOT(object):

    def notify(self, message, event):
-        if not headphones.PUSHALOT_ENABLED:
+        if not headphones.CONFIG.PUSHALOT_ENABLED:
            return

-        pushalot_authorizationtoken = headphones.PUSHALOT_APIKEY
+        pushalot_authorizationtoken = headphones.CONFIG.PUSHALOT_APIKEY

        logger.debug(u"Pushalot event: " + event)
        logger.debug(u"Pushalot message: " + message)
@@ -492,12 +502,12 @@ class PUSHALOT(object):

        data = {'AuthorizationToken': pushalot_authorizationtoken,
                'Title': event.encode('utf-8'),
-                'Body': message.encode("utf-8") }
+                'Body': message.encode("utf-8")}

        http_handler.request("POST",
                                "/api/sendmessage",
-                                headers = {'Content-type': "application/x-www-form-urlencoded"},
-                                body = urlencode(data))
+                                headers={'Content-type': "application/x-www-form-urlencoded"},
+                                body=urlencode(data))
        response = http_handler.getresponse()
        request_status = response.status

@@ -515,6 +525,7 @@ class PUSHALOT(object):
                logger.info(u"Pushalot notification failed.")
                return False

+
 class Synoindex(object):
    def __init__(self, util_loc='/usr/syno/bin/synoindex'):
        self.util_loc = util_loc
@@ -551,15 +562,16 @@ class Synoindex(object):
            for path in path_list:
                self.notify(path)

+
 class PUSHOVER(object):

    def __init__(self):
-        self.enabled = headphones.PUSHOVER_ENABLED
-        self.keys = headphones.PUSHOVER_KEYS
-        self.priority = headphones.PUSHOVER_PRIORITY
+        self.enabled = headphones.CONFIG.PUSHOVER_ENABLED
+        self.keys = headphones.CONFIG.PUSHOVER_KEYS
+        self.priority = headphones.CONFIG.PUSHOVER_PRIORITY

-        if headphones.PUSHOVER_APITOKEN:
-            self.application_token = headphones.PUSHOVER_APITOKEN
+        if headphones.CONFIG.PUSHOVER_APITOKEN:
+            self.application_token = headphones.CONFIG.PUSHOVER_APITOKEN
        else:
            self.application_token = "LdPCoy0dqC21ktsbEyAVCcwvQiVlsz"

@@ -567,36 +579,27 @@ class PUSHOVER(object):
        return cherrypy.config['config'].get('Pushover', options)

    def notify(self, message, event):
-        if not headphones.PUSHOVER_ENABLED:
+        if not headphones.CONFIG.PUSHOVER_ENABLED:
            return

-        http_handler = HTTPSConnection("api.pushover.net")
+        url = "https://api.pushover.net/1/messages.json"

        data = {'token': self.application_token,
-                'user': headphones.PUSHOVER_KEYS,
+                'user': headphones.CONFIG.PUSHOVER_KEYS,
                'title': event,
                'message': message.encode("utf-8"),
-                'priority': headphones.PUSHOVER_PRIORITY }
+                'priority': headphones.CONFIG.PUSHOVER_PRIORITY}

-        http_handler.request("POST",
-                                "/1/messages.json",
-                                headers = {'Content-type': "application/x-www-form-urlencoded"},
-                                body = urlencode(data))
-        response = http_handler.getresponse()
-        request_status = response.status
-        logger.debug(u"Pushover response status: %r" % request_status)
-        logger.debug(u"Pushover response headers: %r" % response.getheaders())
-        logger.debug(u"Pushover response body: %r" % response.read())
+        headers = {'Content-type': "application/x-www-form-urlencoded"}

-        if request_status == 200:
-                logger.info(u"Pushover notifications sent.")
-                return True
-        elif request_status >= 400 and request_status < 500:
-                logger.info(u"Pushover request failed: %s" % response.reason)
-                return False
+        response = request.request_response(url, method="POST", headers=headers, data=data)
+
+        if response:
+            logger.info(u"Pushover notifications sent.")
+            return True
        else:
-                logger.info(u"Pushover notification failed.")
-                return False
+            logger.error(u"Pushover notification failed.")
+            return False

    def updateLibrary(self):
        #For uniformity reasons not removed
@@ -609,33 +612,33 @@ class PUSHOVER(object):

        self.notify('Main Screen Activate', 'Test Message')

+
 class TwitterNotifier(object):

    REQUEST_TOKEN_URL = 'https://api.twitter.com/oauth/request_token'
-    ACCESS_TOKEN_URL  = 'https://api.twitter.com/oauth/access_token'
+    ACCESS_TOKEN_URL = 'https://api.twitter.com/oauth/access_token'
    AUTHORIZATION_URL = 'https://api.twitter.com/oauth/authorize'
-    SIGNIN_URL        = 'https://api.twitter.com/oauth/authenticate'
+    SIGNIN_URL = 'https://api.twitter.com/oauth/authenticate'

    def __init__(self):
        self.consumer_key = "oYKnp2ddX5gbARjqX8ZAAg"
        self.consumer_secret = "A4Xkw9i5SjHbTk7XT8zzOPqivhj9MmRDR9Qn95YA9sk"

    def notify_snatch(self, title):
-        if headphones.TWITTER_ONSNATCH:
-            self._notifyTwitter(common.notifyStrings[common.NOTIFY_SNATCH]+': '+title+' at '+helpers.now())
+        if headphones.CONFIG.TWITTER_ONSNATCH:
+            self._notifyTwitter(common.notifyStrings[common.NOTIFY_SNATCH] + ': ' + title + ' at ' + helpers.now())

    def notify_download(self, title):
-        if headphones.TWITTER_ENABLED:
-            self._notifyTwitter(common.notifyStrings[common.NOTIFY_DOWNLOAD]+': '+title+' at '+helpers.now())
+        if headphones.CONFIG.TWITTER_ENABLED:
+            self._notifyTwitter(common.notifyStrings[common.NOTIFY_DOWNLOAD] + ': ' + title + ' at ' + helpers.now())

    def test_notify(self):
-        return self._notifyTwitter("This is a test notification from Headphones at "+helpers.now(), force=True)
+        return self._notifyTwitter("This is a test notification from Headphones at " + helpers.now(), force=True)

    def _get_authorization(self):

-        signature_method_hmac_sha1 = oauth.SignatureMethod_HMAC_SHA1() #@UnusedVariable
-        oauth_consumer             = oauth.Consumer(key=self.consumer_key, secret=self.consumer_secret)
-        oauth_client               = oauth.Client(oauth_consumer)
+        oauth_consumer = oauth.Consumer(key=self.consumer_key, secret=self.consumer_secret)
+        oauth_client = oauth.Client(oauth_consumer)

        logger.info('Requesting temp token from Twitter')

@@ -646,95 +649,105 @@ class TwitterNotifier(object):
        else:
            request_token = dict(parse_qsl(content))

-            headphones.TWITTER_USERNAME = request_token['oauth_token']
-            headphones.TWITTER_PASSWORD = request_token['oauth_token_secret']
+            headphones.CONFIG.TWITTER_USERNAME = request_token['oauth_token']
+            headphones.CONFIG.TWITTER_PASSWORD = request_token['oauth_token_secret']

-            return self.AUTHORIZATION_URL+"?oauth_token="+ request_token['oauth_token']
+            return self.AUTHORIZATION_URL + "?oauth_token=" + request_token['oauth_token']

    def _get_credentials(self, key):
        request_token = {}

-        request_token['oauth_token'] = headphones.TWITTER_USERNAME
-        request_token['oauth_token_secret'] = headphones.TWITTER_PASSWORD
+        request_token['oauth_token'] = headphones.CONFIG.TWITTER_USERNAME
+        request_token['oauth_token_secret'] = headphones.CONFIG.TWITTER_PASSWORD
        request_token['oauth_callback_confirmed'] = 'true'

        token = oauth.Token(request_token['oauth_token'], request_token['oauth_token_secret'])
        token.set_verifier(key)

-        logger.info('Generating and signing request for an access token using key '+key)
+        logger.info('Generating and signing request for an access token using key ' + key)

-        signature_method_hmac_sha1 = oauth.SignatureMethod_HMAC_SHA1() #@UnusedVariable
-        oauth_consumer             = oauth.Consumer(key=self.consumer_key, secret=self.consumer_secret)
-        logger.info('oauth_consumer: '+str(oauth_consumer))
-        oauth_client  = oauth.Client(oauth_consumer, token)
-        logger.info('oauth_client: '+str(oauth_client))
+        oauth_consumer = oauth.Consumer(key=self.consumer_key, secret=self.consumer_secret)
+        logger.info('oauth_consumer: ' + str(oauth_consumer))
+        oauth_client = oauth.Client(oauth_consumer, token)
+        logger.info('oauth_client: ' + str(oauth_client))
        resp, content = oauth_client.request(self.ACCESS_TOKEN_URL, method='POST', body='oauth_verifier=%s' % key)
-        logger.info('resp, content: '+str(resp)+','+str(content))
+        logger.info('resp, content: ' + str(resp) + ',' + str(content))

-        access_token  = dict(parse_qsl(content))
-        logger.info('access_token: '+str(access_token))
+        access_token = dict(parse_qsl(content))
+        logger.info('access_token: ' + str(access_token))

-        logger.info('resp[status] = '+str(resp['status']))
+        logger.info('resp[status] = ' + str(resp['status']))
        if resp['status'] != '200':
-            logger.info('The request for a token with did not succeed: '+str(resp['status']), logger.ERROR)
+            logger.info('The request for a token with did not succeed: ' + str(resp['status']), logger.ERROR)
            return False
        else:
            logger.info('Your Twitter Access Token key: %s' % access_token['oauth_token'])
            logger.info('Access Token secret: %s' % access_token['oauth_token_secret'])
-            headphones.TWITTER_USERNAME = access_token['oauth_token']
-            headphones.TWITTER_PASSWORD = access_token['oauth_token_secret']
+            headphones.CONFIG.TWITTER_USERNAME = access_token['oauth_token']
+            headphones.CONFIG.TWITTER_PASSWORD = access_token['oauth_token_secret']
            return True

-
    def _send_tweet(self, message=None):

-        username=self.consumer_key
-        password=self.consumer_secret
-        access_token_key=headphones.TWITTER_USERNAME
-        access_token_secret=headphones.TWITTER_PASSWORD
+        username = self.consumer_key
+        password = self.consumer_secret
+        access_token_key = headphones.CONFIG.TWITTER_USERNAME
+        access_token_secret = headphones.CONFIG.TWITTER_PASSWORD

-        logger.info(u"Sending tweet: "+message)
+        logger.info(u"Sending tweet: " + message)

        api = twitter.Api(username, password, access_token_key, access_token_secret)

        try:
            api.PostUpdate(message)
-        except Exception, e:
+        except Exception as e:
            logger.info(u"Error Sending Tweet: %s" % e)
            return False

        return True

    def _notifyTwitter(self, message='', force=False):
-        prefix = headphones.TWITTER_PREFIX
+        prefix = headphones.CONFIG.TWITTER_PREFIX

-        if not headphones.TWITTER_ENABLED and not force:
+        if not headphones.CONFIG.TWITTER_ENABLED and not force:
            return False

-        return self._send_tweet(prefix+": "+message)
+        return self._send_tweet(prefix + ": " + message)
+

 class OSX_NOTIFY(object):

    def __init__(self):
        try:
            self.objc = __import__("objc")
+            self.AppKit = __import__("AppKit")
        except:
+            logger.warn('OS X Notification: Cannot import objc or AppKit')
            return False

    def swizzle(self, cls, SEL, func):
-        old_IMP = cls.instanceMethodForSelector_(SEL)
+        old_IMP = getattr(cls, SEL, None)
+        if old_IMP is None:
+            old_IMP = cls.instanceMethodForSelector_(SEL)
+
        def wrapper(self, *args, **kwargs):
            return func(self, old_IMP, *args, **kwargs)
-        new_IMP = self.objc.selector(wrapper, selector=old_IMP.selector,
-            signature=old_IMP.signature)
-        self.objc.classAddMethod(cls, SEL, new_IMP)

-    def notify(self, title, subtitle=None, text=None, sound=True):
+        new_IMP = self.objc.selector(
+            wrapper,
+            selector=old_IMP.selector,
+            signature=old_IMP.signature
+        )
+        self.objc.classAddMethod(cls, SEL.encode(), new_IMP)
+
+    def notify(self, title, subtitle=None, text=None, sound=True, image=None):

        try:
-            self.swizzle(self.objc.lookUpClass('NSBundle'),
-                b'bundleIdentifier',
-                self.swizzled_bundleIdentifier)
+            self.swizzle(
+                self.objc.lookUpClass('NSBundle'),
+                'bundleIdentifier',
+                self.swizzled_bundleIdentifier
+            )

            NSUserNotification = self.objc.lookUpClass('NSUserNotification')
            NSUserNotificationCenter = self.objc.lookUpClass('NSUserNotificationCenter')
@@ -753,6 +766,10 @@ class OSX_NOTIFY(object):
                notification.setInformativeText_(text)
            if sound:
                notification.setSoundName_("NSUserNotificationDefaultSoundName")
+            if image:
+                source_img = self.AppKit.NSImage.alloc().initByReferencingFile_(image)
+                notification.setContentImage_(source_img)
+                #notification.set_identityImage_(source_img)
            notification.setHasActionButton_(False)

            notification_center = NSUserNotificationCenter.defaultUserNotificationCenter()
@@ -761,13 +778,14 @@ class OSX_NOTIFY(object):
            del pool
            return True

-        except Exception, e:
+        except Exception as e:
            logger.warn('Error sending OS X Notification: %s' % e)
            return False

    def swizzled_bundleIdentifier(self, original, swizzled):
        return 'ade.headphones.osxnotify'

+
 class BOXCAR(object):

    def __init__(self):
@@ -779,7 +797,7 @@ class BOXCAR(object):
                message += '<br></br><a href="http://musicbrainz.org/release-group/%s">MusicBrainz</a>' % rgid

            data = urllib.urlencode({
-                'user_credentials': headphones.BOXCAR_TOKEN,
+                'user_credentials': headphones.CONFIG.BOXCAR_TOKEN,
                'notification[title]': title.encode('utf-8'),
                'notification[long_message]': message.encode('utf-8'),
                'notification[sound]': "done"
@@ -794,12 +812,13 @@ class BOXCAR(object):
            logger.warn('Error sending Boxcar2 Notification: %s' % e)
            return False

+
 class SubSonicNotifier(object):

    def __init__(self):
-        self.host = headphones.SUBSONIC_HOST
-        self.username = headphones.SUBSONIC_USERNAME
-        self.password = headphones.SUBSONIC_PASSWORD
+        self.host = headphones.CONFIG.SUBSONIC_HOST
+        self.username = headphones.CONFIG.SUBSONIC_USERNAME
+        self.password = headphones.CONFIG.SUBSONIC_PASSWORD

    def notify(self, albumpaths):
        # Correct URL
@@ -811,4 +830,35 @@ class SubSonicNotifier(object):

        # Invoke request
        request.request_response(self.host + "musicFolderSettings.view?scanNow",
-            auth=(self.username, self.password))
+            auth=(self.username, self.password))
+
+class Email(object):
+
+    def notify(self, subject, message):
+
+        message = MIMEText(message, 'plain', "utf-8")
+        message['Subject'] = subject
+        message['From'] = email.utils.formataddr(('Headphones', headphones.CONFIG.EMAIL_FROM))
+        message['To'] = headphones.CONFIG.EMAIL_TO
+
+        try:
+            if (headphones.CONFIG.EMAIL_SSL):
+                mailserver = smtplib.SMTP_SSL(headphones.CONFIG.EMAIL_SMTP_SERVER, headphones.CONFIG.EMAIL_SMTP_PORT)
+            else:
+                mailserver = smtplib.SMTP(headphones.CONFIG.EMAIL_SMTP_SERVER, headphones.CONFIG.EMAIL_SMTP_PORT)
+
+            if (headphones.CONFIG.EMAIL_TLS):
+                mailserver.starttls()
+
+            mailserver.ehlo()
+
+            if headphones.CONFIG.EMAIL_SMTP_USER:
+                mailserver.login(headphones.CONFIG.EMAIL_SMTP_USER, headphones.CONFIG.EMAIL_SMTP_PASSWORD)
+
+            mailserver.sendmail(headphones.CONFIG.EMAIL_FROM, headphones.CONFIG.EMAIL_TO, message.as_string())
+            mailserver.quit()
+            return True
+
+        except Exception, e:
+            logger.warn('Error sending Email: %s' % e)
+            return False
@@ -19,37 +19,33 @@
 # along with Sick Beard.  If not, see <http://www.gnu.org/licenses/>.


-
 import httplib
-import datetime

 import headphones

 from base64 import standard_b64encode
 import xmlrpclib

-#from headphones.providers.generic import GenericProvider
-
 from headphones import logger

+
 def sendNZB(nzb):

    addToTop = False
-    nzbgetXMLrpc = "%(username)s:%(password)s@%(host)s/xmlrpc"
+    nzbgetXMLrpc = "%(protocol)s://%(username)s:%(password)s@%(host)s/xmlrpc"

-    if headphones.NZBGET_HOST == None:
+    if not headphones.CONFIG.NZBGET_HOST:
        logger.error(u"No NZBget host found in configuration. Please configure it.")
        return False

-    if headphones.NZBGET_HOST.startswith('https://'):
-        nzbgetXMLrpc = 'https://' + nzbgetXMLrpc
-        headphones.NZBGET_HOST.replace('https://','',1)
+    if headphones.CONFIG.NZBGET_HOST.startswith('https://'):
+        protocol = 'https'
+        host = headphones.CONFIG.NZBGET_HOST.replace('https://', '', 1)
    else:
-        nzbgetXMLrpc = 'http://' + nzbgetXMLrpc
-        headphones.NZBGET_HOST.replace('http://','',1)
+        protocol = 'http'
+        host = headphones.CONFIG.NZBGET_HOST.replace('http://', '', 1)

-
-    url = nzbgetXMLrpc % {"host": headphones.NZBGET_HOST, "username": headphones.NZBGET_USERNAME, "password": headphones.NZBGET_PASSWORD}
+    url = nzbgetXMLrpc % {"protocol": protocol, "host": host, "username": headphones.CONFIG.NZBGET_USERNAME, "password": headphones.CONFIG.NZBGET_PASSWORD}

    nzbGetRPC = xmlrpclib.ServerProxy(url)
    try:
@@ -63,7 +59,7 @@ def sendNZB(nzb):
        return False

    except xmlrpclib.ProtocolError, e:
-        if (e.errmsg == "Unauthorized"):
+        if e.errmsg == "Unauthorized":
            logger.error(u"NZBget password is incorrect.")
        else:
            logger.error(u"Protocol Error: " + e.errmsg)
@@ -86,35 +82,37 @@ def sendNZB(nzb):
        nzbget_version = int(nzbget_version_str[:nzbget_version_str.find(".")])
        if nzbget_version == 0:
            if nzbcontent64 is not None:
-                nzbget_result = nzbGetRPC.append(nzb.name + ".nzb", headphones.NZBGET_CATEGORY, addToTop, nzbcontent64)
+                nzbget_result = nzbGetRPC.append(nzb.name + ".nzb", headphones.CONFIG.NZBGET_CATEGORY, addToTop, nzbcontent64)
            else:
-                if nzb.resultType == "nzb":
-                    genProvider = GenericProvider("")
-                    data = genProvider.getURL(nzb.url)
-                    if (data == None):
-                        return False
-                    nzbcontent64 = standard_b64encode(data)
-                nzbget_result = nzbGetRPC.append(nzb.name + ".nzb", headphones.NZBGET_CATEGORY, addToTop, nzbcontent64)
+                # from headphones.common.providers.generic import GenericProvider
+                # if nzb.resultType == "nzb":
+                #     genProvider = GenericProvider("")
+                #     data = genProvider.getURL(nzb.url)
+                #     if (data is None):
+                #         return False
+                #     nzbcontent64 = standard_b64encode(data)
+                # nzbget_result = nzbGetRPC.append(nzb.name + ".nzb", headphones.CONFIG.NZBGET_CATEGORY, addToTop, nzbcontent64)
+                return False
        elif nzbget_version == 12:
            if nzbcontent64 is not None:
-                nzbget_result = nzbGetRPC.append(nzb.name + ".nzb", headphones.NZBGET_CATEGORY, headphones.NZBGET_PRIORITY, False,
+                nzbget_result = nzbGetRPC.append(nzb.name + ".nzb", headphones.CONFIG.NZBGET_CATEGORY, headphones.CONFIG.NZBGET_PRIORITY, False,
                                                 nzbcontent64, False, dupekey, dupescore, "score")
            else:
-                nzbget_result = nzbGetRPC.appendurl(nzb.name + ".nzb", headphones.NZBGET_CATEGORY, headphones.NZBGET_PRIORITY, False,
+                nzbget_result = nzbGetRPC.appendurl(nzb.name + ".nzb", headphones.CONFIG.NZBGET_CATEGORY, headphones.CONFIG.NZBGET_PRIORITY, False,
                                                    nzb.url, False, dupekey, dupescore, "score")
        # v13+ has a new combined append method that accepts both (url and content)
        # also the return value has changed from boolean to integer
        # (Positive number representing NZBID of the queue item. 0 and negative numbers represent error codes.)
        elif nzbget_version >= 13:
            nzbget_result = True if nzbGetRPC.append(nzb.name + ".nzb", nzbcontent64 if nzbcontent64 is not None else nzb.url,
-                                                     headphones.NZBGET_CATEGORY, headphones.NZBGET_PRIORITY, False, False, dupekey, dupescore,
+                                                     headphones.CONFIG.NZBGET_CATEGORY, headphones.CONFIG.NZBGET_PRIORITY, False, False, dupekey, dupescore,
                                                     "score") > 0 else False
        else:
            if nzbcontent64 is not None:
-                nzbget_result = nzbGetRPC.append(nzb.name + ".nzb", headphones.NZBGET_CATEGORY, headphones.NZBGET_PRIORITY, False,
+                nzbget_result = nzbGetRPC.append(nzb.name + ".nzb", headphones.CONFIG.NZBGET_CATEGORY, headphones.CONFIG.NZBGET_PRIORITY, False,
                                                 nzbcontent64)
            else:
-                nzbget_result = nzbGetRPC.appendurl(nzb.name + ".nzb", headphones.NZBGET_CATEGORY, headphones.NZBGET_PRIORITY, False,
+                nzbget_result = nzbGetRPC.appendurl(nzb.name + ".nzb", headphones.CONFIG.NZBGET_CATEGORY, headphones.CONFIG.NZBGET_PRIORITY, False,
                                                    nzb.url)

        if nzbget_result:
@@ -21,13 +21,26 @@ from bs4 import BeautifulSoup
 import requests
 import feedparser
 import headphones
+import headphones.lock
+import collections
+
+# Dictionary with last request times, for rate limiting.
+last_requests = collections.defaultdict(int)
+fake_lock = headphones.lock.FakeLock()
+

 def request_response(url, method="get", auto_raise=True,
-    whitelist_status_code=None, **kwargs):
+                     whitelist_status_code=None, lock=fake_lock, **kwargs):
    """
-    Convenient wrapper for `requests.get', which will capture the exceptions and
-    log them. On success, the Response object is returned. In case of a
+    Convenient wrapper for `requests.get', which will capture the exceptions
+    and log them. On success, the Response object is returned. In case of a
    exception, None is returned.
+
+    Additionally, there is support for rate limiting. To use this feature,
+    supply a tuple of (lock, request_limit). The lock is used to make sure no
+    other request with the same lock is executed. The request limit is the
+    minimal time between two requests (and so 1/request_limit is the number of
+    requests per seconds).
    """

    # Convert whitelist_status_code to a list if needed
@@ -36,16 +49,18 @@ def request_response(url, method="get", auto_raise=True,

    # Disable verification of SSL certificates if requested. Note: this could
    # pose a security issue!
-    kwargs["verify"] = headphones.VERIFY_SSL_CERT
+    kwargs["verify"] = bool(headphones.CONFIG.VERIFY_SSL_CERT)

-    # Map method to the request.XXX method. This is a simple hack, but it allows
-    # requests to apply more magic per method. See lib/requests/api.py.
+    # Map method to the request.XXX method. This is a simple hack, but it
+    # allows requests to apply more magic per method. See lib/requests/api.py.
    request_method = getattr(requests, method.lower())

    try:
-        # Request the URL
-        logger.debug("Requesting URL via %s method: %s", method.upper(), url)
-        response = request_method(url, **kwargs)
+        # Request URL and wait for response
+        with lock:
+            logger.debug(
+                "Requesting URL via %s method: %s", method.upper(), url)
+            response = request_method(url, **kwargs)

        # If status code != OK, then raise exception, except if the status code
        # is white listed.
@@ -54,19 +69,33 @@ def request_response(url, method="get", auto_raise=True,
                try:
                    response.raise_for_status()
                except:
-                    logger.debug("Response status code %d is not white " \
-                         "listed, raised exception", response.status_code)
+                    logger.debug(
+                        "Response status code %d is not white "
+                        "listed, raised exception", response.status_code)
                    raise
        elif auto_raise:
            response.raise_for_status()

        return response
+    except requests.exceptions.SSLError as e:
+        if kwargs["verify"]:
+            logger.error(
+                "Unable to connect to remote host because of a SSL error. "
+                "It is likely that your system cannot verify the validity"
+                "of the certificate. The remote certificate is either "
+                "self-signed, or the remote server uses SNI. See the wiki for "
+                "more information on this topic.")
+        else:
+            logger.error(
+                "SSL error raised during connection, with certificate "
+                "verification turned off: %s", e)
    except requests.ConnectionError:
-        logger.error("Unable to connect to remote host. Check if the remote " \
+        logger.error(
+            "Unable to connect to remote host. Check if the remote "
            "host is up and running.")
    except requests.Timeout:
-        logger.error("Request timed out. The remote host did not respeond " \
-            "timely.")
+        logger.error(
+            "Request timed out. The remote host did not respond in a timely manner.")
    except requests.HTTPError as e:
        if e.response is not None:
            if e.response.status_code >= 500:
@@ -77,7 +106,8 @@ def request_response(url, method="get", auto_raise=True,
                # I don't think we will end up here, but for completeness
                cause = "unknown"

-            logger.error("Request raise HTTP error with status code %d (%s).",
+            logger.error(
+                "Request raise HTTP error with status code %d (%s).",
                e.response.status_code, cause)

            # Debug response
@@ -88,6 +118,7 @@ def request_response(url, method="get", auto_raise=True,
    except requests.RequestException as e:
        logger.error("Request raised exception: %s", e)

+
 def request_soup(url, **kwargs):
    """
    Wrapper for `request_response', which will return a BeatifulSoup object if
@@ -100,6 +131,7 @@ def request_soup(url, **kwargs):
    if response is not None:
        return BeautifulSoup(response.content, parser)

+
 def request_minidom(url, **kwargs):
    """
    Wrapper for `request_response', which will return a Minidom object if no
@@ -111,13 +143,14 @@ def request_minidom(url, **kwargs):
    if response is not None:
        return minidom.parseString(response.content)

+
 def request_json(url, **kwargs):
    """
    Wrapper for `request_response', which will decode the response as JSON
    object and return the result, if no exceptions are raised.

-    As an option, a validator callback can be given, which should return True if
-    the result is valid.
+    As an option, a validator callback can be given, which should return True
+    if the result is valid.
    """

    validator = kwargs.pop("validator", None)
@@ -138,6 +171,7 @@ def request_json(url, **kwargs):
            if headphones.VERBOSE:
                server_message(response)

+
 def request_content(url, **kwargs):
    """
    Wrapper for `request_response', which will return the raw content.
@@ -148,6 +182,7 @@ def request_content(url, **kwargs):
    if response is not None:
        return response.content

+
 def request_feed(url, **kwargs):
    """
    Wrapper for `request_response', which will return a feed object.
@@ -158,6 +193,7 @@ def request_feed(url, **kwargs):
    if response is not None:
        return feedparser.parse(response.content)

+
 def server_message(response):
    """
    Extract server message from response and log in to logger with DEBUG level.
@@ -179,9 +215,9 @@ def server_message(response):
        # Find body and cleanup common tags to grab content, which probably
        # contains the message.
        message = soup.find("body")
+        elements = ("header", "script", "footer", "nav", "input", "textarea")

-        for element in ["header", "script", "footer", "nav", "input",
-            "textarea"]:
+        for element in elements:

            for tag in soup.find_all(element):
                tag.replaceWith("")
@@ -198,4 +234,4 @@ def server_message(response):
        if len(message) > 150:
            message = message[:150] + "..."

-        logger.debug("Server responded with message: %s", message)
+        logger.debug("Server responded with message: %s", message)
@@ -19,37 +19,54 @@

 import MultipartPostHandler
 import headphones
-import datetime
 import cookielib
-import urllib2
 import httplib
-import urllib
-import ast

 from headphones.common import USER_AGENT
-from headphones import logger
-from headphones import notifiers, helpers
+from headphones import logger, helpers, request
+
+
+def sab_api_call(request_type=None, params={}, **kwargs):
+
+    if not headphones.CONFIG.SAB_HOST.startswith('http'):
+        headphones.CONFIG.SAB_HOST = 'http://' + headphones.CONFIG.SAB_HOST
+
+    if headphones.CONFIG.SAB_HOST.endswith('/'):
+        headphones.CONFIG.SAB_HOST = headphones.CONFIG.SAB_HOST[0:len(headphones.CONFIG.SAB_HOST) - 1]
+
+    url = headphones.CONFIG.SAB_HOST + "/" + "api?"
+
+    if headphones.CONFIG.SAB_USERNAME:
+        params['ma_username'] = headphones.CONFIG.SAB_USERNAME
+    if headphones.CONFIG.SAB_PASSWORD:
+        params['ma_password'] = headphones.CONFIG.SAB_PASSWORD
+    if headphones.CONFIG.SAB_APIKEY:
+        params['apikey'] = headphones.CONFIG.SAB_APIKEY
+    
+    if request_type=='send_nzb' and headphones.CONFIG.SAB_CATEGORY:
+        params['cat'] = headphones.CONFIG.SAB_CATEGORY
+
+    params['output']='json'
+
+    response = request.request_json(url, params=params, **kwargs)
+
+    if not response:
+        logger.error("Error connecting to SABnzbd on url: %s" % headphones.CONFIG.SAB_HOST)
+        return False
+    else:
+        logger.debug("Successfully connected to SABnzbd on url: %s" % headphones.CONFIG.SAB_HOST)
+        return response

 def sendNZB(nzb):

    params = {}
-
-    if headphones.SAB_USERNAME:
-        params['ma_username'] = headphones.SAB_USERNAME
-    if headphones.SAB_PASSWORD:
-        params['ma_password'] = headphones.SAB_PASSWORD
-    if headphones.SAB_APIKEY:
-        params['apikey'] = headphones.SAB_APIKEY
-    if headphones.SAB_CATEGORY:
-        params['cat'] = headphones.SAB_CATEGORY
-
    # if it's a normal result we just pass SAB the URL
    if nzb.resultType == "nzb":
        # for newzbin results send the ID to sab specifically
        if nzb.provider.getID() == 'newzbin':
            id = nzb.provider.getIDFromURL(nzb.url)
            if not id:
-                logger.info("Unable to send NZB to sab, can't find ID in URL "+str(nzb.url))
+                logger.info("Unable to send NZB to sab, can't find ID in URL " + str(nzb.url))
                return False
            params['mode'] = 'addid'
            params['name'] = id
@@ -62,101 +79,41 @@ def sendNZB(nzb):
        # Sanitize the file a bit, since we can only use ascii chars with MultiPartPostHandler
        nzbdata = helpers.latinToAscii(nzb.extraInfo[0])
        params['mode'] = 'addfile'
-        multiPartParams = {"nzbfile": (helpers.latinToAscii(nzb.name)+".nzb", nzbdata)}
+        files = {"nzbfile": (helpers.latinToAscii(nzb.name) + ".nzb", nzbdata)}
+        headers = {'User-Agent': USER_AGENT}

-    if not headphones.SAB_HOST.startswith('http'):
-        headphones.SAB_HOST = 'http://' + headphones.SAB_HOST
+    logger.info("Attempting to connect to SABnzbd on url: %s" % headphones.CONFIG.SAB_HOST)
+    if nzb.resultType == "nzb":
+        response = sab_api_call('send_nzb', params=params)
+    elif nzb.resultType == "nzbdata":
+        cookies = cookielib.CookieJar()
+        response = sab_api_call('send_nzb', params=params, method="post", files=files, cookies=cookies, headers=headers)

-    if headphones.SAB_HOST.endswith('/'):
-        headphones.SAB_HOST = headphones.SAB_HOST[0:len(headphones.SAB_HOST)-1]
-
-    url = headphones.SAB_HOST + "/" + "api?" + urllib.urlencode(params)
-
-    try:
-
-        if nzb.resultType == "nzb":
-            f = urllib.urlopen(url)
-        elif nzb.resultType == "nzbdata":
-            cookies = cookielib.CookieJar()
-            opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookies),
-                                          MultipartPostHandler.MultipartPostHandler)
-
-            req = urllib2.Request(url,
-                                  multiPartParams,
-                                  headers={'User-Agent': USER_AGENT})
-
-            f = opener.open(req)
-
-    except (EOFError, IOError), e:
-        logger.error(u"Unable to connect to SAB with URL: %s" % url)
-        return False
-
-    except httplib.InvalidURL, e:
-        logger.error(u"Invalid SAB host, check your config. Current host: %s" % headphones.SAB_HOST)
-        return False
-        
-    except Exception, e:
-        logger.error(u"Error: " + str(e))
-        return False
-        
-    if f == None:
+    if not response:
        logger.info(u"No data returned from SABnzbd, NZB not sent")
        return False

-    try:
-        result = f.readlines()
-    except Exception, e:
-        logger.info(u"Error trying to get result from SAB, NZB not sent: ")
-        return False
-
-    if len(result) == 0:
-        logger.info(u"No data returned from SABnzbd, NZB not sent")
-        return False
-
-    sabText = result[0].strip()
-
-    logger.info(u"Result text from SAB: " + sabText)
-
-    if sabText == "ok":
-        logger.info(u"NZB sent to SAB successfully")
+    if response['status']:
+        logger.info(u"NZB sent to SABnzbd successfully")
        return True
-    elif sabText == "Missing authentication":
-        logger.info(u"Incorrect username/password sent to SAB, NZB not sent")
-        return False
    else:
-        logger.info(u"Unknown failure sending NZB to sab. Return text is: " + sabText)
+        logger.error(u"Error sending NZB to SABnzbd: %s" % response['error'])
        return False
-        
+
+
 def checkConfig():

-    params = { 'mode' : 'get_config', 
-               'section' : 'misc' 
+    params = {'mode': 'get_config',
+               'section': 'misc',
               }

-    if headphones.SAB_USERNAME:
-        params['ma_username'] = headphones.SAB_USERNAME
-    if headphones.SAB_PASSWORD:
-        params['ma_password'] = headphones.SAB_PASSWORD
-    if headphones.SAB_APIKEY:
-        params['apikey'] = headphones.SAB_APIKEY
-
-    if not headphones.SAB_HOST.startswith('http'):
-        headphones.SAB_HOST = 'http://' + headphones.SAB_HOST
-
-    if headphones.SAB_HOST.endswith('/'):
-        headphones.SAB_HOST = headphones.SAB_HOST[0:len(headphones.SAB_HOST)-1]
+    config_options = sab_api_call(params=params)
    
-    url = headphones.SAB_HOST + "/" + "api?" + urllib.urlencode(params)
-    
-    try:
-        f = urllib.urlopen(url).read()
-    except Exception, e:
+    if not config_options:
        logger.warn("Unable to read SABnzbd config file - cannot determine renaming options (might affect auto & forced post processing)")
        return (0, 0)
-        
-    config_options = ast.literal_eval(f)
-    
-    replace_spaces = config_options['misc']['replace_spaces']
-    replace_dots = config_options['misc']['replace_dots']
-    
+
+    replace_spaces = config_options['config']['misc']['replace_spaces']
+    replace_dots = config_options['config']['misc']['replace_dots']
+
    return (replace_spaces, replace_dots)
@@ -4,8 +4,6 @@
 # Headphones rutracker.org search
 # Functions called from searcher.py

-from headphones import logger, db, utorrent
-
 from bencode import bencode as bencode, bdecode
 from urlparse import urlparse
 from bs4 import BeautifulSoup
@@ -20,6 +18,9 @@ import urllib
 import re
 import os

+from headphones import db, logger
+
+
 class Rutracker():

    logged_in = False
@@ -47,9 +48,9 @@ class Rutracker():
        #if self.login_counter > 1:
        #    return False

-        params = urllib.urlencode({"login_username" : login,
-                                   "login_password" : password,
-                                   "login" : "Вход"})
+        params = urllib.urlencode({"login_username": login,
+                                   "login_password": password,
+                                   "login": "Вход"})

        try:
            self.opener.open("http://login.rutracker.org/forum/login.php", params)
@@ -114,26 +115,26 @@ class Rutracker():
            #logger.debug (soup.prettify())

            # Title
-            for link in soup.find_all('a', attrs={'class' : 'med tLink hl-tags bold'}):
+            for link in soup.find_all('a', attrs={'class': 'med tLink hl-tags bold'}):
                title = link.get_text()
                titles.append(title)

            # Download URL
-            for link in soup.find_all('a', attrs={'class' : 'small tr-dl dl-stub'}):
+            for link in soup.find_all('a', attrs={'class': 'small tr-dl dl-stub'}):
                url = link.get('href')
                urls.append(url)

            # Seeders
-            for link in soup.find_all('b', attrs={'class' : 'seedmed'}):
+            for link in soup.find_all('b', attrs={'class': 'seedmed'}):
                seeder = link.get_text()
                seeders.append(seeder)

            # Size
-            for link in soup.find_all('td', attrs={'class' : 'row4 small nowrap tor-size'}):
+            for link in soup.find_all('td', attrs={'class': 'row4 small nowrap tor-size'}):
                size = link.u.string
                sizes.append(size)

-        except :
+        except:
            pass

        # Combine lists
@@ -196,8 +197,8 @@ class Rutracker():
                        if torrent:
                            decoded = bdecode(torrent)
                            metainfo = decoded['info']
-                        page.close ()
-                    except Exception, e:
+                        page.close()
+                    except Exception as e:
                        logger.error('Error getting torrent: %s' % e)
                        return False

@@ -215,9 +216,9 @@ class Rutracker():
                                    cuecount += 1

                    title = returntitle.lower()
-                    logger.debug ('torrent title: %s' % title)
-                    logger.debug ('headphones trackcount: %s' % hptrackcount)
-                    logger.debug ('rutracker trackcount: %s' % trackcount)
+                    logger.debug('torrent title: %s' % title)
+                    logger.debug('headphones trackcount: %s' % hptrackcount)
+                    logger.debug('rutracker trackcount: %s' % trackcount)

                    # If torrent track count less than headphones track count, and there's a cue, then attempt to get track count from log(s)
                    # This is for the case where we have a single .flac/.wav which can be split by cue
@@ -245,7 +246,7 @@ class Rutracker():

                    if totallogcount > 0:
                        trackcount = totallogcount
-                        logger.debug ('rutracker logtrackcount: %s' % totallogcount)
+                        logger.debug('rutracker logtrackcount: %s' % totallogcount)

                    # If torrent track count = hp track count then return torrent,
                    # if greater, check for deluxe/special/foreign editions
@@ -294,7 +295,7 @@ class Rutracker():
            os.umask(prev)

            # Add file to utorrent
-            if headphones.TORRENT_DOWNLOADER == 2:
+            if headphones.CONFIG.TORRENT_DOWNLOADER == 2:
                self.utorrent_add_file(download_path)

        except Exception as e:
@@ -306,7 +307,7 @@ class Rutracker():
    #TODO get this working in utorrent.py
    def utorrent_add_file(self, filename):

-        host = headphones.UTORRENT_HOST
+        host = headphones.CONFIG.UTORRENT_HOST
        if not host.startswith('http'):
            host = 'http://' + host
        if host.endswith('/'):
@@ -315,8 +316,8 @@ class Rutracker():
            host = host[:-4]

        base_url = host
-        username = headphones.UTORRENT_USERNAME
-        password = headphones.UTORRENT_PASSWORD
+        username = headphones.CONFIG.UTORRENT_USERNAME
+        password = headphones.CONFIG.UTORRENT_PASSWORD

        session = requests.Session()
        url = base_url + '/gui/'
@@ -346,4 +347,3 @@ class Rutracker():
            except Exception:
                logger.exception('Error adding file to utorrent')
                return
-
@@ -13,19 +13,21 @@
 #  You should have received a copy of the GNU General Public License
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

+from headphones import db, utorrent, transmission, logger
+
 import threading
 import headphones
-from headphones import db, utorrent, transmission, logger

 postprocessor_lock = threading.Lock()

-# Remove Torrent + data if Post Processed and finished Seeding
 def checkTorrentFinished():
+    """
+    Remove Torrent + data if Post Processed and finished Seeding
+    """

    logger.info("Checking if any torrents have finished seeding and can be removed")

    with postprocessor_lock:
-
        myDB = db.DBConnection()
        results = myDB.select('SELECT * from snatched WHERE Status="Seed_Processed"')

@@ -33,10 +35,12 @@ def checkTorrentFinished():
            hash = album['FolderName']
            albumid = album['AlbumID']
            torrent_removed = False
-            if headphones.TORRENT_DOWNLOADER == 1:
+            if headphones.CONFIG.TORRENT_DOWNLOADER == 1:
                torrent_removed = transmission.removeTorrent(hash, True)
            else:
                torrent_removed = utorrent.removeTorrent(hash, True)

            if torrent_removed:
                myDB.action('DELETE from snatched WHERE status = "Seed_Processed" and AlbumID=?', [albumid])
+
+    logger.info("Checking finished torrents completed")
@@ -13,12 +13,12 @@
 #  You should have received a copy of the GNU General Public License
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

-from headphones import logger, notifiers, request
+from headphones import logger, request

-import re
 import time
 import json
 import base64
+import urlparse
 import headphones

 # This is just a simple script to send torrents to transmission. The
@@ -27,30 +27,28 @@ import headphones
 # TODO: Store the session id so we don't need to make 2 calls
 #       Store torrent id so we can check up on it

+
 def addTorrent(link):
    method = 'torrent-add'

    if link.endswith('.torrent'):
        with open(link, 'rb') as f:
            metainfo = str(base64.b64encode(f.read()))
-        arguments = {'metainfo': metainfo, 'download-dir':headphones.DOWNLOAD_TORRENT_DIR}
+        arguments = {'metainfo': metainfo, 'download-dir': headphones.CONFIG.DOWNLOAD_TORRENT_DIR}
    else:
-        arguments = {'filename': link, 'download-dir': headphones.DOWNLOAD_TORRENT_DIR}
+        arguments = {'filename': link, 'download-dir': headphones.CONFIG.DOWNLOAD_TORRENT_DIR}

-    response = torrentAction(method,arguments)
+    response = torrentAction(method, arguments)

    if not response:
        return False

    if response['result'] == 'success':
        if 'torrent-added' in response['arguments']:
-            name = response['arguments']['torrent-added']['name']
            retid = response['arguments']['torrent-added']['hashString']
        elif 'torrent-duplicate' in response['arguments']:
-            name = response['arguments']['torrent-duplicate']['name']
            retid = response['arguments']['torrent-duplicate']['hashString']
        else:
-            name = link
            retid = False

        logger.info(u"Torrent sent to Transmission successfully")
@@ -60,9 +58,10 @@ def addTorrent(link):
        logger.info('Transmission returned status %s' % response['result'])
        return False

+
 def getTorrentFolder(torrentid):
    method = 'torrent-get'
-    arguments = { 'ids': torrentid, 'fields': ['name','percentDone']}
+    arguments = {'ids': torrentid, 'fields': ['name', 'percentDone']}

    response = torrentAction(method, arguments)
    percentdone = response['arguments']['torrents'][0]['percentDone']
@@ -70,8 +69,8 @@ def getTorrentFolder(torrentid):

    tries = 1

-    while percentdone == 0  and tries <10:
-        tries+=1
+    while percentdone == 0 and tries < 10:
+        tries += 1
        time.sleep(5)
        response = torrentAction(method, arguments)
        percentdone = response['arguments']['torrents'][0]['percentDone']
@@ -80,6 +79,7 @@ def getTorrentFolder(torrentid):

    return torrent_folder_name

+
 def setSeedRatio(torrentid, ratio):
    method = 'torrent-set'
    if ratio != 0:
@@ -91,10 +91,11 @@ def setSeedRatio(torrentid, ratio):
    if not response:
        return False

-def removeTorrent(torrentid, remove_data = False):
+
+def removeTorrent(torrentid, remove_data=False):

    method = 'torrent-get'
-    arguments = { 'ids': torrentid, 'fields': ['isFinished', 'name']}
+    arguments = {'ids': torrentid, 'fields': ['isFinished', 'name']}

    response = torrentAction(method, arguments)
    if not response:
@@ -120,12 +121,12 @@ def removeTorrent(torrentid, remove_data = False):

    return False

+
 def torrentAction(method, arguments):

-    host = headphones.TRANSMISSION_HOST
-    username = headphones.TRANSMISSION_USERNAME
-    password = headphones.TRANSMISSION_PASSWORD
-    sessionid = None
+    host = headphones.CONFIG.TRANSMISSION_HOST
+    username = headphones.CONFIG.TRANSMISSION_USERNAME
+    password = headphones.CONFIG.TRANSMISSION_PASSWORD

    if not host.startswith('http'):
        host = 'http://' + host
@@ -133,28 +134,22 @@ def torrentAction(method, arguments):
    if host.endswith('/'):
        host = host[:-1]

-    # Either the host ends with a port, or some directory, or rpc
-    # If it ends with /rpc we don't have to do anything
-    # If it ends with a port we add /transmission/rpc
-    # anything else we just add rpc
-    if not host.endswith('/rpc'):
-        # Check if it ends in a port number
-        i = host.rfind(':')
-        if i >= 0:
-            possible_port = host[i+1:]
-            try:
-                port = int(possible_port)
-                host = host + "/transmission/rpc"
-            except ValueError:
-                host = host + "/rpc"
-        else:
-            logger.error('Transmission port missing')
-            return
+    # Fix the URL. We assume that the user does not point to the RPC endpoint,
+    # so add it if it is missing.
+    parts = list(urlparse.urlparse(host))
+
+    if not parts[0] in ("http", "https"):
+        parts[0] = "http"
+
+    if not parts[2].endswith("/rpc"):
+        parts[2] += "/transmission/rpc"
+
+    host = urlparse.urlunparse(parts)

    # Retrieve session id
    auth = (username, password) if username and password else None
-
-    response = request.request_response(host, auth=auth, whitelist_status_code=[401, 409])
+    response = request.request_response(host, auth=auth,
+        whitelist_status_code=[401, 409])

    if response is None:
        logger.error("Error gettings Transmission session ID")
@@ -163,23 +158,27 @@ def torrentAction(method, arguments):
    # Parse response
    if response.status_code == 401:
        if auth:
-            logger.error("Username and/or password not accepted by Transmission")
+            logger.error("Username and/or password not accepted by " \
+                "Transmission")
        else:
            logger.error("Transmission authorization required")

        return
    elif response.status_code == 409:
-        sessionid = response.headers['x-transmission-session-id']
+        session_id = response.headers['x-transmission-session-id']

-    if not sessionid:
-        logger.error("Error getting Session ID from Transmission")
-        return
+        if not session_id:
+            logger.error("Expected a Session ID from Transmission")
+            return

    # Prepare next request
-    headers = { 'x-transmission-session-id': sessionid }
-    data = { 'method': method, 'arguments': arguments }
+    headers = {'x-transmission-session-id': session_id}
+    data = {'method': method, 'arguments': arguments}

-    response = request.request_json(host, method="post", data=json.dumps(data), headers=headers, auth=auth)
+    response = request.request_json(host, method="POST", data=json.dumps(data),
+        headers=headers, auth=auth)
+
+    print response

    if not response:
        logger.error("Error sending torrent to Transmission")
@@ -13,19 +13,17 @@
 #  You should have received a copy of the GNU General Public License
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

-import headphones
-
 from headphones import logger, db, importer

+
 def dbUpdate(forcefull=False):

    myDB = db.DBConnection()

-    activeartists = myDB.select('SELECT ArtistID, ArtistName from artists WHERE Status="Active" or Status="Loading" order by LastUpdated ASC')
-    logger.info('Starting update for %i active artists' % len(activeartists))
-
-    for artist in activeartists:
+    active_artists = myDB.select('SELECT ArtistID, ArtistName from artists WHERE Status="Active" or Status="Loading" order by LastUpdated ASC')
+    logger.info('Starting update for %i active artists', len(active_artists))

+    for artist in active_artists:
        artistid = artist[0]
        importer.addArtisttoDB(artistid=artistid, extrasonly=False, forcefull=forcefull)

@@ -13,22 +13,29 @@
 #  You should have received a copy of the GNU General Public License
 #  along with Headphones.  If not, see <http://www.gnu.org/licenses/>.

-import urllib, urllib2, urlparse, cookielib
-import json, re, os, time
+import urllib
+import urllib2
+import urlparse
+import cookielib
+import json
+import re
+import os
+import time

 import headphones

 from headphones import logger
 from collections import namedtuple

+
 class utorrentclient(object):

    TOKEN_REGEX = "<div id='token' style='display:none;'>([^<>]+)</div>"
    UTSetting = namedtuple("UTSetting", ["name", "int", "str", "access"])

-    def __init__(self, base_url = None, username = None, password = None,):
+    def __init__(self, base_url=None, username=None, password=None,):

-        host = headphones.UTORRENT_HOST
+        host = headphones.CONFIG.UTORRENT_HOST
        if not host.startswith('http'):
            host = 'http://' + host

@@ -39,8 +46,8 @@ class utorrentclient(object):
            host = host[:-4]

        self.base_url = host
-        self.username = headphones.UTORRENT_USERNAME
-        self.password = headphones.UTORRENT_PASSWORD
+        self.username = headphones.CONFIG.UTORRENT_USERNAME
+        self.password = headphones.CONFIG.UTORRENT_PASSWORD
        self.opener = self._make_opener('uTorrent', self.base_url, self.username, self.password)
        self.token = self._get_token()
        #TODO refresh token, when necessary
@@ -48,7 +55,7 @@ class utorrentclient(object):
    def _make_opener(self, realm, base_url, username, password):
        """uTorrent API need HTTP Basic Auth and cookie support for token verify."""
        auth = urllib2.HTTPBasicAuthHandler()
-        auth.add_password(realm=realm,uri=base_url,user=username,passwd=password)
+        auth.add_password(realm=realm, uri=base_url, user=username, passwd=password)
        opener = urllib2.build_opener(auth)
        urllib2.install_opener(opener)

@@ -132,7 +139,7 @@ class utorrentclient(object):
            return settings[key]
        return settings

-    def remove(self, hash, remove_data = False):
+    def remove(self, hash, remove_data=False):
        if remove_data:
            params = [('action', 'removedata'), ('hash', hash)]
        else:
@@ -156,13 +163,15 @@ class utorrentclient(object):
            logger.debug('URL: ' + str(url))
            logger.debug('uTorrent webUI raised the following error: ' + str(err))

+
 def labelTorrent(hash):
-    label = headphones.UTORRENT_LABEL
+    label = headphones.CONFIG.UTORRENT_LABEL
    uTorrentClient = utorrentclient()
    if label:
-        uTorrentClient.setprops(hash,'label',label)
+        uTorrentClient.setprops(hash, 'label', label)

-def removeTorrent(hash, remove_data = False):
+
+def removeTorrent(hash, remove_data=False):
    uTorrentClient = utorrentclient()
    status, torrentList = uTorrentClient.list()
    torrents = torrentList['torrents']
@@ -177,14 +186,16 @@ def removeTorrent(hash, remove_data = False):
                return False
    return False

+
 def setSeedRatio(hash, ratio):
    uTorrentClient = utorrentclient()
    uTorrentClient.setprops(hash, 'seed_override', '1')
    if ratio != 0:
-        uTorrentClient.setprops(hash,'seed_ratio', ratio * 10)
+        uTorrentClient.setprops(hash, 'seed_ratio', ratio * 10)
    else:
        # TODO passing -1 should be unlimited
-        uTorrentClient.setprops(hash,'seed_ratio', -10)
+        uTorrentClient.setprops(hash, 'seed_ratio', -10)
+

 def dirTorrent(hash, cacheid=None, return_name=None):

@@ -204,7 +215,7 @@ def dirTorrent(hash, cacheid=None, return_name=None):
    cacheid = torrentList['torrentc']

    for torrent in torrents:
-        if (torrent[0].lower() == hash):
+        if torrent[0].upper() == hash:
            if not return_name:
                return torrent[26], cacheid
            else:
@@ -212,6 +223,7 @@ def dirTorrent(hash, cacheid=None, return_name=None):

    return None, None

+
 def addTorrent(link, hash):
    uTorrentClient = utorrentclient()

@@ -230,7 +242,7 @@ def addTorrent(link, hash):
    # If there's no folder yet then it's probably a magnet, try until folder is populated
    if torrent_folder == active_dir or not torrent_folder:
        tries = 1
-        while (torrent_folder == active_dir or torrent_folder == None) and tries <= 10:
+        while (torrent_folder == active_dir or torrent_folder is None) and tries <= 10:
            tries += 1
            time.sleep(6)
            torrent_folder, cacheid = dirTorrent(hash, cacheid)
@@ -241,8 +253,11 @@ def addTorrent(link, hash):
        return torrent_folder
    else:
        labelTorrent(hash)
+        if headphones.SYS_PLATFORM != "win32":
+            torrent_folder = torrent_folder.replace('\\', '/')
        return os.path.basename(os.path.normpath(torrent_folder))

+
 def getSettingsDirectories():
    uTorrentClient = utorrentclient()
    settings = uTorrentClient.get_settings()
@@ -253,4 +268,3 @@ def getSettingsDirectories():
    if 'dir_completed_download' in settings:
        completed = settings['dir_completed_download'][2]
    return active, completed
-
@@ -1 +1 @@
-HEADPHONES_VERSION = "master"
+HEADPHONES_VERSION = "master"
@@ -22,10 +22,11 @@ import subprocess

 from headphones import logger, version, request

+
 def runGit(args):

-    if headphones.GIT_PATH:
-        git_locations = ['"'+headphones.GIT_PATH+'"']
+    if headphones.CONFIG.GIT_PATH:
+        git_locations = ['"' + headphones.CONFIG.GIT_PATH + '"']
    else:
        git_locations = ['git']

@@ -35,7 +36,7 @@ def runGit(args):
    output = err = None

    for cur_git in git_locations:
-        cmd = cur_git+' '+args
+        cmd = cur_git + ' ' + args

        try:
            logger.debug('Trying to execute: "' + cmd + '" with shell in ' + headphones.PROG_DIR)
@@ -59,6 +60,7 @@ def runGit(args):

    return (output, err)

+
 def getVersion():

    if version.HEADPHONES_VERSION.startswith('win32build'):
@@ -82,16 +84,16 @@ def getVersion():
            logger.error('Output doesn\'t look like a hash, not using it')
            cur_commit_hash = None

-        if headphones.DO_NOT_OVERRIDE_GIT_BRANCH and headphones.GIT_BRANCH:
-            branch_name = headphones.GIT_BRANCH
+        if headphones.CONFIG.DO_NOT_OVERRIDE_GIT_BRANCH and headphones.CONFIG.GIT_BRANCH:
+            branch_name = headphones.CONFIG.GIT_BRANCH

        else:
            branch_name, err = runGit('rev-parse --abbrev-ref HEAD')
            branch_name = branch_name

-            if not branch_name and headphones.GIT_BRANCH:
-                logger.error('Could not retrieve branch name from git. Falling back to %s' % headphones.GIT_BRANCH)
-                branch_name = headphones.GIT_BRANCH
+            if not branch_name and headphones.CONFIG.GIT_BRANCH:
+                logger.error('Could not retrieve branch name from git. Falling back to %s' % headphones.CONFIG.GIT_BRANCH)
+                branch_name = headphones.CONFIG.GIT_BRANCH
            if not branch_name:
                logger.error('Could not retrieve branch name from git. Defaulting to master')
                branch_name = 'master'
@@ -111,16 +113,17 @@ def getVersion():
            current_version = f.read().strip(' \n\r')

        if current_version:
-            return current_version, headphones.GIT_BRANCH
+            return current_version, headphones.CONFIG.GIT_BRANCH
        else:
            return None, 'master'

+
 def checkGithub():
    headphones.COMMITS_BEHIND = 0

    # Get the latest version available from github
    logger.info('Retrieving latest version information from GitHub')
-    url = 'https://api.github.com/repos/%s/headphones/commits/%s' % (headphones.GIT_USER, headphones.GIT_BRANCH)
+    url = 'https://api.github.com/repos/%s/headphones/commits/%s' % (headphones.CONFIG.GIT_USER, headphones.CONFIG.GIT_BRANCH)
    version = request.request_json(url, timeout=20, validator=lambda x: type(x) == dict)

    if version is None:
@@ -140,7 +143,7 @@ def checkGithub():
        return headphones.LATEST_VERSION

    logger.info('Comparing currently installed version with latest GitHub version')
-    url = 'https://api.github.com/repos/%s/headphones/compare/%s...%s' % (headphones.GIT_USER, headphones.LATEST_VERSION, headphones.CURRENT_VERSION)
+    url = 'https://api.github.com/repos/%s/headphones/compare/%s...%s' % (headphones.CONFIG.GIT_USER, headphones.LATEST_VERSION, headphones.CURRENT_VERSION)
    commits = request.request_json(url, timeout=20, whitelist_status_code=404, validator=lambda x: type(x) == dict)

    if commits is None:
@@ -161,12 +164,13 @@ def checkGithub():

    return headphones.LATEST_VERSION

+
 def update():
    if headphones.INSTALL_TYPE == 'win':
        logger.info('Windows .exe updating not supported yet.')

    elif headphones.INSTALL_TYPE == 'git':
-        output, err = runGit('pull origin ' + headphones.GIT_BRANCH)
+        output, err = runGit('pull origin ' + headphones.CONFIG.GIT_BRANCH)

        if not output:
            logger.error('Couldn\'t download latest version')
@@ -177,22 +181,22 @@ def update():
                logger.info('No update available, not updating')
                logger.info('Output: ' + str(output))
            elif line.endswith('Aborting.'):
-                logger.error('Unable to update from git: '+line)
+                logger.error('Unable to update from git: ' + line)
                logger.info('Output: ' + str(output))

    else:
-        tar_download_url = 'https://github.com/%s/headphones/tarball/%s' % (headphones.GIT_USER, headphones.GIT_BRANCH)
+        tar_download_url = 'https://github.com/%s/headphones/tarball/%s' % (headphones.CONFIG.GIT_USER, headphones.CONFIG.GIT_BRANCH)
        update_dir = os.path.join(headphones.PROG_DIR, 'update')
        version_path = os.path.join(headphones.PROG_DIR, 'version.txt')

-        logger.info('Downloading update from: '+ tar_download_url)
+        logger.info('Downloading update from: ' + tar_download_url)
        data = request.request_content(tar_download_url)

        if not data:
            logger.error("Unable to retrieve new version from '%s', can't update", tar_download_url)
            return

-        download_name = headphones.GIT_BRANCH + '-github'
+        download_name = headphones.CONFIG.GIT_BRANCH + '-github'
        tar_download_path = os.path.join(headphones.PROG_DIR, download_name)

        # Save tar to disk
@@ -212,13 +216,13 @@ def update():
        # Find update dir name
        update_dir_contents = [x for x in os.listdir(update_dir) if os.path.isdir(os.path.join(update_dir, x))]
        if len(update_dir_contents) != 1:
-            logger.error("Invalid update data, update failed: "+str(update_dir_contents))
+            logger.error("Invalid update data, update failed: " + str(update_dir_contents))
            return
        content_dir = os.path.join(update_dir, update_dir_contents[0])

        # walk temp folder and move files to main folder
        for dirname, dirnames, filenames in os.walk(content_dir):
-            dirname = dirname[len(content_dir)+1:]
+            dirname = dirname[len(content_dir) + 1:]
            for curfile in filenames:
                old_path = os.path.join(content_dir, dirname, curfile)
                new_path = os.path.join(headphones.PROG_DIR, dirname, curfile)
@@ -232,6 +236,8 @@ def update():
            with open(version_path, 'w') as f:
                f.write(str(headphones.LATEST_VERSION))
        except IOError as e:
-            logger.error("Unable to write current version to version.txt, " \
-                "update not complete: ", e)
+            logger.error(
+                "Unable to write current version to version.txt, update not complete: %s",
+                e
+            )
            return
@@ -22,9 +22,8 @@ from headphones import logger
 from headphones.webserve import WebInterface
 from headphones.helpers import create_https_certificates

-def initialize(options=None):
-    if options is None:
-        options = {}
+
+def initialize(options):

    # HTTPS stuff stolen from sickbeard
    enable_https = options['enable_https']
@@ -32,16 +31,17 @@ def initialize(options=None):
    https_key = options['https_key']

    if enable_https:
-        # If either the HTTPS certificate or key do not exist, make some self-signed ones.
+        # If either the HTTPS certificate or key do not exist, try to make
+        # self-signed ones.
        if not (https_cert and os.path.exists(https_cert)) or not (https_key and os.path.exists(https_key)):
            if not create_https_certificates(https_cert, https_key):
-                logger.warn(u"Unable to create cert/key files, disabling HTTPS")
-                headphones.ENABLE_HTTPS = False
+                logger.warn("Unable to create certificate and key. Disabling " \
+                    "HTTPS")
                enable_https = False

        if not (os.path.exists(https_cert) and os.path.exists(https_key)):
-            logger.warn(u"Disabled HTTPS because of missing CERT and KEY files")
-            headphones.ENABLE_HTTPS = False
+            logger.warn("Disabled HTTPS because of missing certificate and " \
+                "key.")
            enable_https = False

    options_dict = {
@@ -52,7 +52,7 @@ def initialize(options=None):
        'tools.encode.encoding': 'utf-8',
        'tools.decode.on': True,
        'log.screen': False,
-        'engine.autoreload_on': False,
+        'engine.autoreload.on': False,
    }

    if enable_https:
@@ -71,30 +71,30 @@ def initialize(options=None):
            'tools.staticdir.root': os.path.join(headphones.PROG_DIR, 'data'),
            'tools.proxy.on': options['http_proxy']  # pay attention to X-Forwarded-Proto header
        },
-        '/interfaces':{
+        '/interfaces': {
            'tools.staticdir.on': True,
            'tools.staticdir.dir': "interfaces"
        },
-        '/images':{
+        '/images': {
            'tools.staticdir.on': True,
            'tools.staticdir.dir': "images"
        },
-        '/css':{
+        '/css': {
            'tools.staticdir.on': True,
            'tools.staticdir.dir': "css"
        },
-        '/js':{
+        '/js': {
            'tools.staticdir.on': True,
            'tools.staticdir.dir': "js"
        },
-        '/favicon.ico':{
+        '/favicon.ico': {
            'tools.staticfile.on': True,
            'tools.staticfile.filename': os.path.join(os.path.abspath(
                os.curdir), "images" + os.sep + "favicon.ico")
        },
-        '/cache':{
+        '/cache': {
            'tools.staticdir.on': True,
-            'tools.staticdir.dir': headphones.CACHE_DIR
+            'tools.staticdir.dir': headphones.CONFIG.CACHE_DIR
        }
    }

@@ -104,23 +104,21 @@ def initialize(options=None):
        conf['/'].update({
            'tools.auth_basic.on': True,
            'tools.auth_basic.realm': 'Headphones web server',
-            'tools.auth_basic.checkpassword':  cherrypy.lib.auth_basic \
-                .checkpassword_dict({
-                    options['http_username']: options['http_password']
-                })
+            'tools.auth_basic.checkpassword': cherrypy.lib.auth_basic.checkpassword_dict({
+                options['http_username']: options['http_password']
+            })
        })
-        conf['/api'] = { 'tools.auth_basic.on': False }
-
+        conf['/api'] = {'tools.auth_basic.on': False}

    # Prevent time-outs
    cherrypy.engine.timeout_monitor.unsubscribe()
-    cherrypy.tree.mount(WebInterface(), options['http_root'], config=conf)
+    cherrypy.tree.mount(WebInterface(), str(options['http_root']), config=conf)

    try:
-        cherrypy.process.servers.check_port(options['http_host'], options['http_port'])
+        cherrypy.process.servers.check_port(str(options['http_host']), options['http_port'])
        cherrypy.server.start()
    except IOError:
        sys.stderr.write('Failed to start on port: %i. Is something else running?\n' % (options['http_port']))
        sys.exit(1)

-    cherrypy.server.wait()
+    cherrypy.server.wait()
@@ -3,16 +3,16 @@
 <plist version="1.0">
 <dict>
  <key>Label</key>
-  <string>com.headphones.headphones</string>
+  <string>headphones</string>
  <key>ProgramArguments</key>
  <array>
+       <!-- Modify these two lines if you need to to reflect your python location and Headphones install location -->
       <string>/usr/bin/python</string>
-       <string>/Applications/Headphones/headphones.py</string>
-       <string>--quiet</string>
-       <string>--daemon</string>
-       <string>--nolaunch</string>
+       <string>/Applications/Headphones/Headphones.py</string>
  </array>
  <key>RunAtLoad</key>
  <true/>
+  <key>KeepAlive</key>
+  <true/>
 </dict>
-</plist>
+</plist>
@@ -1,3 +1,5 @@
-version_info = (2, 0, 0, 'rc', 2)
-version = '.'.join(str(n) for n in version_info[:3])
-release = version + ''.join(str(n) for n in version_info[3:])
+version_info = (3, 0, 1)
+version = '3.0.1'
+release = '3.0.1'
+
+__version__ = release  # PEP 396
@@ -1,63 +1,72 @@
-__all__ = ('EVENT_SCHEDULER_START', 'EVENT_SCHEDULER_SHUTDOWN',
-           'EVENT_JOBSTORE_ADDED', 'EVENT_JOBSTORE_REMOVED',
-           'EVENT_JOBSTORE_JOB_ADDED', 'EVENT_JOBSTORE_JOB_REMOVED',
-           'EVENT_JOB_EXECUTED', 'EVENT_JOB_ERROR', 'EVENT_JOB_MISSED',
-           'EVENT_ALL', 'SchedulerEvent', 'JobStoreEvent', 'JobEvent')
+__all__ = ('EVENT_SCHEDULER_START', 'EVENT_SCHEDULER_SHUTDOWN', 'EVENT_EXECUTOR_ADDED', 'EVENT_EXECUTOR_REMOVED',
+           'EVENT_JOBSTORE_ADDED', 'EVENT_JOBSTORE_REMOVED', 'EVENT_ALL_JOBS_REMOVED', 'EVENT_JOB_ADDED',
+           'EVENT_JOB_REMOVED', 'EVENT_JOB_MODIFIED', 'EVENT_JOB_EXECUTED', 'EVENT_JOB_ERROR', 'EVENT_JOB_MISSED',
+           'SchedulerEvent', 'JobEvent', 'JobExecutionEvent')


-EVENT_SCHEDULER_START = 1        # The scheduler was started
-EVENT_SCHEDULER_SHUTDOWN = 2     # The scheduler was shut down
-EVENT_JOBSTORE_ADDED = 4         # A job store was added to the scheduler
-EVENT_JOBSTORE_REMOVED = 8       # A job store was removed from the scheduler
-EVENT_JOBSTORE_JOB_ADDED = 16    # A job was added to a job store
-EVENT_JOBSTORE_JOB_REMOVED = 32  # A job was removed from a job store
-EVENT_JOB_EXECUTED = 64          # A job was executed successfully
-EVENT_JOB_ERROR = 128            # A job raised an exception during execution
-EVENT_JOB_MISSED = 256           # A job's execution was missed
-EVENT_ALL = (EVENT_SCHEDULER_START | EVENT_SCHEDULER_SHUTDOWN |
-             EVENT_JOBSTORE_ADDED | EVENT_JOBSTORE_REMOVED |
-             EVENT_JOBSTORE_JOB_ADDED | EVENT_JOBSTORE_JOB_REMOVED |
-             EVENT_JOB_EXECUTED | EVENT_JOB_ERROR | EVENT_JOB_MISSED)
+EVENT_SCHEDULER_START = 1
+EVENT_SCHEDULER_SHUTDOWN = 2
+EVENT_EXECUTOR_ADDED = 4
+EVENT_EXECUTOR_REMOVED = 8
+EVENT_JOBSTORE_ADDED = 16
+EVENT_JOBSTORE_REMOVED = 32
+EVENT_ALL_JOBS_REMOVED = 64
+EVENT_JOB_ADDED = 128
+EVENT_JOB_REMOVED = 256
+EVENT_JOB_MODIFIED = 512
+EVENT_JOB_EXECUTED = 1024
+EVENT_JOB_ERROR = 2048
+EVENT_JOB_MISSED = 4096
+EVENT_ALL = (EVENT_SCHEDULER_START | EVENT_SCHEDULER_SHUTDOWN | EVENT_JOBSTORE_ADDED | EVENT_JOBSTORE_REMOVED |
+             EVENT_JOB_ADDED | EVENT_JOB_REMOVED | EVENT_JOB_MODIFIED | EVENT_JOB_EXECUTED |
+             EVENT_JOB_ERROR | EVENT_JOB_MISSED)


 class SchedulerEvent(object):
    """
    An event that concerns the scheduler itself.

-    :var code: the type code of this event
+    :ivar code: the type code of this event
+    :ivar alias: alias of the job store or executor that was added or removed (if applicable)
    """
-    def __init__(self, code):
+
+    def __init__(self, code, alias=None):
+        super(SchedulerEvent, self).__init__()
        self.code = code
-
-
-class JobStoreEvent(SchedulerEvent):
-    """
-    An event that concerns job stores.
-
-    :var alias: the alias of the job store involved
-    :var job: the new job if a job was added
-    """
-    def __init__(self, code, alias, job=None):
-        SchedulerEvent.__init__(self, code)
        self.alias = alias
-        if job:
-            self.job = job
+
+    def __repr__(self):
+        return '<%s (code=%d)>' % (self.__class__.__name__, self.code)


 class JobEvent(SchedulerEvent):
+    """
+    An event that concerns a job.
+
+    :ivar code: the type code of this event
+    :ivar job_id: identifier of the job in question
+    :ivar jobstore: alias of the job store containing the job in question
+    """
+
+    def __init__(self, code, job_id, jobstore):
+        super(JobEvent, self).__init__(code)
+        self.code = code
+        self.job_id = job_id
+        self.jobstore = jobstore
+
+
+class JobExecutionEvent(JobEvent):
    """
    An event that concerns the execution of individual jobs.

-    :var job: the job instance in question
-    :var scheduled_run_time: the time when the job was scheduled to be run
-    :var retval: the return value of the successfully executed job
-    :var exception: the exception raised by the job
-    :var traceback: the traceback object associated with the exception
+    :ivar scheduled_run_time: the time when the job was scheduled to be run
+    :ivar retval: the return value of the successfully executed job
+    :ivar exception: the exception raised by the job
+    :ivar traceback: a formatted traceback for the exception
    """
-    def __init__(self, code, job, scheduled_run_time, retval=None,
-                 exception=None, traceback=None):
-        SchedulerEvent.__init__(self, code)
-        self.job = job
+
+    def __init__(self, code, job_id, jobstore, scheduled_run_time, retval=None, exception=None, traceback=None):
+        super(JobExecutionEvent, self).__init__(code, job_id, jobstore)
        self.scheduled_run_time = scheduled_run_time
        self.retval = retval
        self.exception = exception
@@ -0,0 +1,28 @@
+from __future__ import absolute_import
+import sys
+
+from apscheduler.executors.base import BaseExecutor, run_job
+
+
+class AsyncIOExecutor(BaseExecutor):
+    """
+    Runs jobs in the default executor of the event loop.
+
+    Plugin alias: ``asyncio``
+    """
+
+    def start(self, scheduler, alias):
+        super(AsyncIOExecutor, self).start(scheduler, alias)
+        self._eventloop = scheduler._eventloop
+
+    def _do_submit_job(self, job, run_times):
+        def callback(f):
+            try:
+                events = f.result()
+            except:
+                self._run_job_error(job.id, *sys.exc_info()[1:])
+            else:
+                self._run_job_success(job.id, events)
+
+        f = self._eventloop.run_in_executor(None, run_job, job, job._jobstore_alias, run_times, self._logger.name)
+        f.add_done_callback(callback)
@@ -0,0 +1,119 @@
+from abc import ABCMeta, abstractmethod
+from collections import defaultdict
+from datetime import datetime, timedelta
+from traceback import format_tb
+import logging
+import sys
+
+from pytz import utc
+import six
+
+from apscheduler.events import JobExecutionEvent, EVENT_JOB_MISSED, EVENT_JOB_ERROR, EVENT_JOB_EXECUTED
+
+
+class MaxInstancesReachedError(Exception):
+    def __init__(self, job):
+        super(MaxInstancesReachedError, self).__init__(
+            'Job "%s" has already reached its maximum number of instances (%d)' % (job.id, job.max_instances))
+
+
+class BaseExecutor(six.with_metaclass(ABCMeta, object)):
+    """Abstract base class that defines the interface that every executor must implement."""
+
+    _scheduler = None
+    _lock = None
+    _logger = logging.getLogger('apscheduler.executors')
+
+    def __init__(self):
+        super(BaseExecutor, self).__init__()
+        self._instances = defaultdict(lambda: 0)
+
+    def start(self, scheduler, alias):
+        """
+        Called by the scheduler when the scheduler is being started or when the executor is being added to an already
+        running scheduler.
+
+        :param apscheduler.schedulers.base.BaseScheduler scheduler: the scheduler that is starting this executor
+        :param str|unicode alias: alias of this executor as it was assigned to the scheduler
+        """
+
+        self._scheduler = scheduler
+        self._lock = scheduler._create_lock()
+        self._logger = logging.getLogger('apscheduler.executors.%s' % alias)
+
+    def shutdown(self, wait=True):
+        """
+        Shuts down this executor.
+
+        :param bool wait: ``True`` to wait until all submitted jobs have been executed
+        """
+
+    def submit_job(self, job, run_times):
+        """
+        Submits job for execution.
+
+        :param Job job: job to execute
+        :param list[datetime] run_times: list of datetimes specifying when the job should have been run
+        :raises MaxInstancesReachedError: if the maximum number of allowed instances for this job has been reached
+        """
+
+        assert self._lock is not None, 'This executor has not been started yet'
+        with self._lock:
+            if self._instances[job.id] >= job.max_instances:
+                raise MaxInstancesReachedError(job)
+
+            self._do_submit_job(job, run_times)
+            self._instances[job.id] += 1
+
+    @abstractmethod
+    def _do_submit_job(self, job, run_times):
+        """Performs the actual task of scheduling `run_job` to be called."""
+
+    def _run_job_success(self, job_id, events):
+        """Called by the executor with the list of generated events when `run_job` has been successfully called."""
+
+        with self._lock:
+            self._instances[job_id] -= 1
+
+        for event in events:
+            self._scheduler._dispatch_event(event)
+
+    def _run_job_error(self, job_id, exc, traceback=None):
+        """Called by the executor with the exception if there is an error calling `run_job`."""
+
+        with self._lock:
+            self._instances[job_id] -= 1
+
+        exc_info = (exc.__class__, exc, traceback)
+        self._logger.error('Error running job %s', job_id, exc_info=exc_info)
+
+
+def run_job(job, jobstore_alias, run_times, logger_name):
+    """Called by executors to run the job. Returns a list of scheduler events to be dispatched by the scheduler."""
+
+    events = []
+    logger = logging.getLogger(logger_name)
+    for run_time in run_times:
+        # See if the job missed its run time window, and handle possible misfires accordingly
+        if job.misfire_grace_time is not None:
+            difference = datetime.now(utc) - run_time
+            grace_time = timedelta(seconds=job.misfire_grace_time)
+            if difference > grace_time:
+                events.append(JobExecutionEvent(EVENT_JOB_MISSED, job.id, jobstore_alias, run_time))
+                logger.warning('Run time of job "%s" was missed by %s', job, difference)
+                continue
+
+        logger.info('Running job "%s" (scheduled at %s)', job, run_time)
+        try:
+            retval = job.func(*job.args, **job.kwargs)
+        except:
+            exc, tb = sys.exc_info()[1:]
+            formatted_tb = ''.join(format_tb(tb))
+            events.append(JobExecutionEvent(EVENT_JOB_ERROR, job.id, jobstore_alias, run_time, exception=exc,
+                                            traceback=formatted_tb))
+            logger.exception('Job "%s" raised an exception', job)
+        else:
+            events.append(JobExecutionEvent(EVENT_JOB_EXECUTED, job.id, jobstore_alias, run_time, retval=retval))
+            logger.info('Job "%s" executed successfully', job)
+
+    return events
@@ -0,0 +1,19 @@
+import sys
+
+from apscheduler.executors.base import BaseExecutor, run_job
+
+
+class DebugExecutor(BaseExecutor):
+    """
+    A special executor that executes the target callable directly instead of deferring it to a thread or process.
+
+    Plugin alias: ``debug``
+    """
+
+    def _do_submit_job(self, job, run_times):
+        try:
+            events = run_job(job, job._jobstore_alias, run_times, self._logger.name)
+        except:
+            self._run_job_error(job.id, *sys.exc_info()[1:])
+        else:
+            self._run_job_success(job.id, events)
@@ -0,0 +1,29 @@
+from __future__ import absolute_import
+import sys
+
+from apscheduler.executors.base import BaseExecutor, run_job
+
+
+try:
+    import gevent
+except ImportError:  # pragma: nocover
+    raise ImportError('GeventExecutor requires gevent installed')
+
+
+class GeventExecutor(BaseExecutor):
+    """
+    Runs jobs as greenlets.
+
+    Plugin alias: ``gevent``
+    """
+
+    def _do_submit_job(self, job, run_times):
+        def callback(greenlet):
+            try:
+                events = greenlet.get()
+            except:
+                self._run_job_error(job.id, *sys.exc_info()[1:])
+            else:
+                self._run_job_success(job.id, events)
+
+        gevent.spawn(run_job, job, job._jobstore_alias, run_times, self._logger.name).link(callback)
@@ -0,0 +1,54 @@
+from abc import abstractmethod
+import concurrent.futures
+
+from apscheduler.executors.base import BaseExecutor, run_job
+
+
+class BasePoolExecutor(BaseExecutor):
+    @abstractmethod
+    def __init__(self, pool):
+        super(BasePoolExecutor, self).__init__()
+        self._pool = pool
+
+    def _do_submit_job(self, job, run_times):
+        def callback(f):
+            exc, tb = (f.exception_info() if hasattr(f, 'exception_info') else
+                       (f.exception(), getattr(f.exception(), '__traceback__', None)))
+            if exc:
+                self._run_job_error(job.id, exc, tb)
+            else:
+                self._run_job_success(job.id, f.result())
+
+        f = self._pool.submit(run_job, job, job._jobstore_alias, run_times, self._logger.name)
+        f.add_done_callback(callback)
+
+    def shutdown(self, wait=True):
+        self._pool.shutdown(wait)
+
+
+class ThreadPoolExecutor(BasePoolExecutor):
+    """
+    An executor that runs jobs in a concurrent.futures thread pool.
+
+    Plugin alias: ``threadpool``
+
+    :param max_workers: the maximum number of spawned threads.
+    """
+
+    def __init__(self, max_workers=10):
+        pool = concurrent.futures.ThreadPoolExecutor(int(max_workers))
+        super(ThreadPoolExecutor, self).__init__(pool)
+
+
+class ProcessPoolExecutor(BasePoolExecutor):
+    """
+    An executor that runs jobs in a concurrent.futures process pool.
+
+    Plugin alias: ``processpool``
+
+    :param max_workers: the maximum number of spawned processes.
+    """
+
+    def __init__(self, max_workers=10):
+        pool = concurrent.futures.ProcessPoolExecutor(int(max_workers))
+        super(ProcessPoolExecutor, self).__init__(pool)
@@ -0,0 +1,25 @@
+from __future__ import absolute_import
+
+from apscheduler.executors.base import BaseExecutor, run_job
+
+
+class TwistedExecutor(BaseExecutor):
+    """
+    Runs jobs in the reactor's thread pool.
+
+    Plugin alias: ``twisted``
+    """
+
+    def start(self, scheduler, alias):
+        super(TwistedExecutor, self).start(scheduler, alias)
+        self._reactor = scheduler._reactor
+
+    def _do_submit_job(self, job, run_times):
+        def callback(success, result):
+            if success:
+                self._run_job_success(job.id, result)
+            else:
+                self._run_job_error(job.id, result.value, result.tb)
+
+        self._reactor.getThreadPool().callInThreadWithCallback(callback, run_job, job, job._jobstore_alias, run_times,
+                                                               self._logger.name)
@@ -1,134 +1,252 @@
-"""
-Jobs represent scheduled tasks.
-"""
+from collections import Iterable, Mapping
+from uuid import uuid4

-from threading import Lock
-from datetime import timedelta
+import six

-from apscheduler.util import to_unicode, ref_to_obj, get_callable_name,\
-    obj_to_ref
-
-
-class MaxInstancesReachedError(Exception):
-    pass
+from apscheduler.triggers.base import BaseTrigger
+from apscheduler.util import ref_to_obj, obj_to_ref, datetime_repr, repr_escape, get_callable_name, check_callable_args, \
+    convert_to_datetime


 class Job(object):
    """
-    Encapsulates the actual Job along with its metadata. Job instances
-    are created by the scheduler when adding jobs, and it should not be
-    directly instantiated.
+    Contains the options given when scheduling callables and its current schedule and other state.
+    This class should never be instantiated by the user.

-    :param trigger: trigger that determines the execution times
-    :param func: callable to call when the trigger is triggered
-    :param args: list of positional arguments to call func with
-    :param kwargs: dict of keyword arguments to call func with
-    :param name: name of the job (optional)
-    :param misfire_grace_time: seconds after the designated run time that
-        the job is still allowed to be run
-    :param coalesce: run once instead of many times if the scheduler determines
-        that the job should be run more than once in succession
-    :param max_runs: maximum number of times this job is allowed to be
-        triggered
-    :param max_instances: maximum number of concurrently running
-        instances allowed for this job
+    :var str id: the unique identifier of this job
+    :var str name: the description of this job
+    :var func: the callable to execute
+    :var tuple|list args: positional arguments to the callable
+    :var dict kwargs: keyword arguments to the callable
+    :var bool coalesce: whether to only run the job once when several run times are due
+    :var trigger: the trigger object that controls the schedule of this job
+    :var str executor: the name of the executor that will run this job
+    :var int misfire_grace_time: the time (in seconds) how much this job's execution is allowed to be late
+    :var int max_instances: the maximum number of concurrently executing instances allowed for this job
+    :var datetime.datetime next_run_time: the next scheduled run time of this job
    """
-    id = None
-    next_run_time = None

-    def __init__(self, trigger, func, args, kwargs, misfire_grace_time,
-                 coalesce, name=None, max_runs=None, max_instances=1):
-        if not trigger:
-            raise ValueError('The trigger must not be None')
-        if not hasattr(func, '__call__'):
-            raise TypeError('func must be callable')
-        if not hasattr(args, '__getitem__'):
-            raise TypeError('args must be a list-like object')
-        if not hasattr(kwargs, '__getitem__'):
-            raise TypeError('kwargs must be a dict-like object')
-        if misfire_grace_time <= 0:
-            raise ValueError('misfire_grace_time must be a positive value')
-        if max_runs is not None and max_runs <= 0:
-            raise ValueError('max_runs must be a positive value')
-        if max_instances <= 0:
-            raise ValueError('max_instances must be a positive value')
+    __slots__ = ('_scheduler', '_jobstore_alias', 'id', 'trigger', 'executor', 'func', 'func_ref', 'args', 'kwargs',
+                 'name', 'misfire_grace_time', 'coalesce', 'max_instances', 'next_run_time')

-        self._lock = Lock()
+    def __init__(self, scheduler, id=None, **kwargs):
+        super(Job, self).__init__()
+        self._scheduler = scheduler
+        self._jobstore_alias = None
+        self._modify(id=id or uuid4().hex, **kwargs)

-        self.trigger = trigger
-        self.func = func
-        self.args = args
-        self.kwargs = kwargs
-        self.name = to_unicode(name or get_callable_name(func))
-        self.misfire_grace_time = misfire_grace_time
-        self.coalesce = coalesce
-        self.max_runs = max_runs
-        self.max_instances = max_instances
-        self.runs = 0
-        self.instances = 0
-
-    def compute_next_run_time(self, now):
-        if self.runs == self.max_runs:
-            self.next_run_time = None
-        else:
-            self.next_run_time = self.trigger.get_next_fire_time(now)
-
-        return self.next_run_time
-
-    def get_run_times(self, now):
+    def modify(self, **changes):
        """
-        Computes the scheduled run times between ``next_run_time`` and ``now``.
+        Makes the given changes to this job and saves it in the associated job store.
+        Accepted keyword arguments are the same as the variables on this class.
+
+        .. seealso:: :meth:`~apscheduler.schedulers.base.BaseScheduler.modify_job`
        """
+
+        self._scheduler.modify_job(self.id, self._jobstore_alias, **changes)
+
+    def reschedule(self, trigger, **trigger_args):
+        """
+        Shortcut for switching the trigger on this job.
+
+        .. seealso:: :meth:`~apscheduler.schedulers.base.BaseScheduler.reschedule_job`
+        """
+
+        self._scheduler.reschedule_job(self.id, self._jobstore_alias, trigger, **trigger_args)
+
+    def pause(self):
+        """
+        Temporarily suspend the execution of this job.
+
+        .. seealso:: :meth:`~apscheduler.schedulers.base.BaseScheduler.pause_job`
+        """
+
+        self._scheduler.pause_job(self.id, self._jobstore_alias)
+
+    def resume(self):
+        """
+        Resume the schedule of this job if previously paused.
+
+        .. seealso:: :meth:`~apscheduler.schedulers.base.BaseScheduler.resume_job`
+        """
+
+        self._scheduler.resume_job(self.id, self._jobstore_alias)
+
+    def remove(self):
+        """
+        Unschedules this job and removes it from its associated job store.
+
+        .. seealso:: :meth:`~apscheduler.schedulers.base.BaseScheduler.remove_job`
+        """
+
+        self._scheduler.remove_job(self.id, self._jobstore_alias)
+
+    @property
+    def pending(self):
+        """Returns ``True`` if the referenced job is still waiting to be added to its designated job store."""
+
+        return self._jobstore_alias is None
+
+    #
+    # Private API
+    #
+
+    def _get_run_times(self, now):
+        """
+        Computes the scheduled run times between ``next_run_time`` and ``now`` (inclusive).
+
+        :type now: datetime.datetime
+        :rtype: list[datetime.datetime]
+        """
+
        run_times = []
-        run_time = self.next_run_time
-        increment = timedelta(microseconds=1)
-        while ((not self.max_runs or self.runs < self.max_runs) and
-               run_time and run_time <= now):
-            run_times.append(run_time)
-            run_time = self.trigger.get_next_fire_time(run_time + increment)
+        next_run_time = self.next_run_time
+        while next_run_time and next_run_time <= now:
+            run_times.append(next_run_time)
+            next_run_time = self.trigger.get_next_fire_time(next_run_time, now)

        return run_times

-    def add_instance(self):
-        self._lock.acquire()
-        try:
-            if self.instances == self.max_instances:
-                raise MaxInstancesReachedError
-            self.instances += 1
-        finally:
-            self._lock.release()
+    def _modify(self, **changes):
+        """Validates the changes to the Job and makes the modifications if and only if all of them validate."""

-    def remove_instance(self):
-        self._lock.acquire()
-        try:
-            assert self.instances > 0, 'Already at 0 instances'
-            self.instances -= 1
-        finally:
-            self._lock.release()
+        approved = {}
+
+        if 'id' in changes:
+            value = changes.pop('id')
+            if not isinstance(value, six.string_types):
+                raise TypeError("id must be a nonempty string")
+            if hasattr(self, 'id'):
+                raise ValueError('The job ID may not be changed')
+            approved['id'] = value
+
+        if 'func' in changes or 'args' in changes or 'kwargs' in changes:
+            func = changes.pop('func') if 'func' in changes else self.func
+            args = changes.pop('args') if 'args' in changes else self.args
+            kwargs = changes.pop('kwargs') if 'kwargs' in changes else self.kwargs
+
+            if isinstance(func, str):
+                func_ref = func
+                func = ref_to_obj(func)
+            elif callable(func):
+                try:
+                    func_ref = obj_to_ref(func)
+                except ValueError:
+                    # If this happens, this Job won't be serializable
+                    func_ref = None
+            else:
+                raise TypeError('func must be a callable or a textual reference to one')
+
+            if not hasattr(self, 'name') and changes.get('name', None) is None:
+                changes['name'] = get_callable_name(func)
+
+            if isinstance(args, six.string_types) or not isinstance(args, Iterable):
+                raise TypeError('args must be a non-string iterable')
+            if isinstance(kwargs, six.string_types) or not isinstance(kwargs, Mapping):
+                raise TypeError('kwargs must be a dict-like object')
+
+            check_callable_args(func, args, kwargs)
+
+            approved['func'] = func
+            approved['func_ref'] = func_ref
+            approved['args'] = args
+            approved['kwargs'] = kwargs
+
+        if 'name' in changes:
+            value = changes.pop('name')
+            if not value or not isinstance(value, six.string_types):
+                raise TypeError("name must be a nonempty string")
+            approved['name'] = value
+
+        if 'misfire_grace_time' in changes:
+            value = changes.pop('misfire_grace_time')
+            if value is not None and (not isinstance(value, six.integer_types) or value <= 0):
+                raise TypeError('misfire_grace_time must be either None or a positive integer')
+            approved['misfire_grace_time'] = value
+
+        if 'coalesce' in changes:
+            value = bool(changes.pop('coalesce'))
+            approved['coalesce'] = value
+
+        if 'max_instances' in changes:
+            value = changes.pop('max_instances')
+            if not isinstance(value, six.integer_types) or value <= 0:
+                raise TypeError('max_instances must be a positive integer')
+            approved['max_instances'] = value
+
+        if 'trigger' in changes:
+            trigger = changes.pop('trigger')
+            if not isinstance(trigger, BaseTrigger):
+                raise TypeError('Expected a trigger instance, got %s instead' % trigger.__class__.__name__)
+
+            approved['trigger'] = trigger
+
+        if 'executor' in changes:
+            value = changes.pop('executor')
+            if not isinstance(value, six.string_types):
+                raise TypeError('executor must be a string')
+            approved['executor'] = value
+
+        if 'next_run_time' in changes:
+            value = changes.pop('next_run_time')
+            approved['next_run_time'] = convert_to_datetime(value, self._scheduler.timezone, 'next_run_time')
+
+        if changes:
+            raise AttributeError('The following are not modifiable attributes of Job: %s' % ', '.join(changes))
+
+        for key, value in six.iteritems(approved):
+            setattr(self, key, value)

    def __getstate__(self):
-        # Prevents the unwanted pickling of transient or unpicklable variables
-        state = self.__dict__.copy()
-        state.pop('instances', None)
-        state.pop('func', None)
-        state.pop('_lock', None)
-        state['func_ref'] = obj_to_ref(self.func)
-        return state
+        # Don't allow this Job to be serialized if the function reference could not be determined
+        if not self.func_ref:
+            raise ValueError('This Job cannot be serialized since the reference to its callable (%r) could not be '
+                             'determined. Consider giving a textual reference (module:function name) instead.' %
+                             (self.func,))
+
+        return {
+            'version': 1,
+            'id': self.id,
+            'func': self.func_ref,
+            'trigger': self.trigger,
+            'executor': self.executor,
+            'args': self.args,
+            'kwargs': self.kwargs,
+            'name': self.name,
+            'misfire_grace_time': self.misfire_grace_time,
+            'coalesce': self.coalesce,
+            'max_instances': self.max_instances,
+            'next_run_time': self.next_run_time
+        }

    def __setstate__(self, state):
-        state['instances'] = 0
-        state['func'] = ref_to_obj(state.pop('func_ref'))
-        state['_lock'] = Lock()
-        self.__dict__ = state
+        if state.get('version', 1) > 1:
+            raise ValueError('Job has version %s, but only version 1 can be handled' % state['version'])
+
+        self.id = state['id']
+        self.func_ref = state['func']
+        self.func = ref_to_obj(self.func_ref)
+        self.trigger = state['trigger']
+        self.executor = state['executor']
+        self.args = state['args']
+        self.kwargs = state['kwargs']
+        self.name = state['name']
+        self.misfire_grace_time = state['misfire_grace_time']
+        self.coalesce = state['coalesce']
+        self.max_instances = state['max_instances']
+        self.next_run_time = state['next_run_time']

    def __eq__(self, other):
        if isinstance(other, Job):
-            return self.id is not None and other.id == self.id or self is other
+            return self.id == other.id
        return NotImplemented

    def __repr__(self):
-        return '<Job (name=%s, trigger=%s)>' % (self.name, repr(self.trigger))
+        return '<Job (id=%s name=%s)>' % (repr_escape(self.id), repr_escape(self.name))

    def __str__(self):
-        return '%s (trigger: %s, next run at: %s)' % (self.name,
-            str(self.trigger), str(self.next_run_time))
+        return '%s (trigger: %s, next run at: %s)' % (repr_escape(self.name), repr_escape(str(self.trigger)),
+                                                      datetime_repr(self.next_run_time))
+
+    def __unicode__(self):
+        return six.u('%s (trigger: %s, next run at: %s)') % (self.name, self.trigger, datetime_repr(self.next_run_time))
@@ -1,25 +1,127 @@
-"""
-Abstract base class that provides the interface needed by all job stores.
-Job store methods are also documented here.
-"""
+from abc import ABCMeta, abstractmethod
+import logging
+
+import six


-class JobStore(object):
-    def add_job(self, job):
-        """Adds the given job from this store."""
-        raise NotImplementedError
+class JobLookupError(KeyError):
+    """Raised when the job store cannot find a job for update or removal."""

-    def update_job(self, job):
-        """Persists the running state of the given job."""
-        raise NotImplementedError
+    def __init__(self, job_id):
+        super(JobLookupError, self).__init__(six.u('No job by the id of %s was found') % job_id)

-    def remove_job(self, job):
-        """Removes the given jobs from this store."""
-        raise NotImplementedError

-    def load_jobs(self):
-        """Loads jobs from this store into memory."""
-        raise NotImplementedError
+class ConflictingIdError(KeyError):
+    """Raised when the uniqueness of job IDs is being violated."""

-    def close(self):
+    def __init__(self, job_id):
+        super(ConflictingIdError, self).__init__(six.u('Job identifier (%s) conflicts with an existing job') % job_id)
+
+
+class TransientJobError(ValueError):
+    """Raised when an attempt to add transient (with no func_ref) job to a persistent job store is detected."""
+
+    def __init__(self, job_id):
+        super(TransientJobError, self).__init__(
+            six.u('Job (%s) cannot be added to this job store because a reference to the callable could not be '
+                  'determined.') % job_id)
+
+
+class BaseJobStore(six.with_metaclass(ABCMeta)):
+    """Abstract base class that defines the interface that every job store must implement."""
+
+    _scheduler = None
+    _alias = None
+    _logger = logging.getLogger('apscheduler.jobstores')
+
+    def start(self, scheduler, alias):
+        """
+        Called by the scheduler when the scheduler is being started or when the job store is being added to an already
+        running scheduler.
+
+        :param apscheduler.schedulers.base.BaseScheduler scheduler: the scheduler that is starting this job store
+        :param str|unicode alias: alias of this job store as it was assigned to the scheduler
+        """
+
+        self._scheduler = scheduler
+        self._alias = alias
+        self._logger = logging.getLogger('apscheduler.jobstores.%s' % alias)
+
+    def shutdown(self):
        """Frees any resources still bound to this job store."""
+
+    @abstractmethod
+    def lookup_job(self, job_id):
+        """
+        Returns a specific job, or ``None`` if it isn't found..
+
+        The job store is responsible for setting the ``scheduler`` and ``jobstore`` attributes of the returned job to
+        point to the scheduler and itself, respectively.
+
+        :param str|unicode job_id: identifier of the job
+        :rtype: Job
+        """
+
+    @abstractmethod
+    def get_due_jobs(self, now):
+        """
+        Returns the list of jobs that have ``next_run_time`` earlier or equal to ``now``.
+        The returned jobs must be sorted by next run time (ascending).
+
+        :param datetime.datetime now: the current (timezone aware) datetime
+        :rtype: list[Job]
+        """
+
+    @abstractmethod
+    def get_next_run_time(self):
+        """
+        Returns the earliest run time of all the jobs stored in this job store, or ``None`` if there are no active jobs.
+
+        :rtype: datetime.datetime
+        """
+
+    @abstractmethod
+    def get_all_jobs(self):
+        """
+        Returns a list of all jobs in this job store. The returned jobs should be sorted by next run time (ascending).
+        Paused jobs (next_run_time is None) should be sorted last.
+
+        The job store is responsible for setting the ``scheduler`` and ``jobstore`` attributes of the returned jobs to
+        point to the scheduler and itself, respectively.
+
+        :rtype: list[Job]
+        """
+
+    @abstractmethod
+    def add_job(self, job):
+        """
+        Adds the given job to this store.
+
+        :param Job job: the job to add
+        :raises ConflictingIdError: if there is another job in this store with the same ID
+        """
+
+    @abstractmethod
+    def update_job(self, job):
+        """
+        Replaces the job in the store with the given newer version.
+
+        :param Job job: the job to update
+        :raises JobLookupError: if the job does not exist
+        """
+
+    @abstractmethod
+    def remove_job(self, job_id):
+        """
+        Removes the given job from this store.
+
+        :param str|unicode job_id: identifier of the job
+        :raises JobLookupError: if the job does not exist
+        """
+
+    @abstractmethod
+    def remove_all_jobs(self):
+        """Removes all jobs from this store."""
+
+    def __repr__(self):
+        return '<%s>' % self.__class__.__name__
@@ -0,0 +1,107 @@
+from __future__ import absolute_import
+
+from apscheduler.jobstores.base import BaseJobStore, JobLookupError, ConflictingIdError
+from apscheduler.util import datetime_to_utc_timestamp
+
+
+class MemoryJobStore(BaseJobStore):
+    """
+    Stores jobs in an array in RAM. Provides no persistence support.
+
+    Plugin alias: ``memory``
+    """
+
+    def __init__(self):
+        super(MemoryJobStore, self).__init__()
+        self._jobs = []  # list of (job, timestamp), sorted by next_run_time and job id (ascending)
+        self._jobs_index = {}  # id -> (job, timestamp) lookup table
+
+    def lookup_job(self, job_id):
+        return self._jobs_index.get(job_id, (None, None))[0]
+
+    def get_due_jobs(self, now):
+        now_timestamp = datetime_to_utc_timestamp(now)
+        pending = []
+        for job, timestamp in self._jobs:
+            if timestamp is None or timestamp > now_timestamp:
+                break
+            pending.append(job)
+
+        return pending
+
+    def get_next_run_time(self):
+        return self._jobs[0][0].next_run_time if self._jobs else None
+
+    def get_all_jobs(self):
+        return [j[0] for j in self._jobs]
+
+    def add_job(self, job):
+        if job.id in self._jobs_index:
+            raise ConflictingIdError(job.id)
+
+        timestamp = datetime_to_utc_timestamp(job.next_run_time)
+        index = self._get_job_index(timestamp, job.id)
+        self._jobs.insert(index, (job, timestamp))
+        self._jobs_index[job.id] = (job, timestamp)
+
+    def update_job(self, job):
+        old_job, old_timestamp = self._jobs_index.get(job.id, (None, None))
+        if old_job is None:
+            raise JobLookupError(job.id)
+
+        # If the next run time has not changed, simply replace the job in its present index.
+        # Otherwise, reinsert the job to the list to preserve the ordering.
+        old_index = self._get_job_index(old_timestamp, old_job.id)
+        new_timestamp = datetime_to_utc_timestamp(job.next_run_time)
+        if old_timestamp == new_timestamp:
+            self._jobs[old_index] = (job, new_timestamp)
+        else:
+            del self._jobs[old_index]
+            new_index = self._get_job_index(new_timestamp, job.id)
+            self._jobs.insert(new_index, (job, new_timestamp))
+
+        self._jobs_index[old_job.id] = (job, new_timestamp)
+
+    def remove_job(self, job_id):
+        job, timestamp = self._jobs_index.get(job_id, (None, None))
+        if job is None:
+            raise JobLookupError(job_id)
+
+        index = self._get_job_index(timestamp, job_id)
+        del self._jobs[index]
+        del self._jobs_index[job.id]
+
+    def remove_all_jobs(self):
+        self._jobs = []
+        self._jobs_index = {}
+
+    def shutdown(self):
+        self.remove_all_jobs()
+
+    def _get_job_index(self, timestamp, job_id):
+        """
+        Returns the index of the given job, or if it's not found, the index where the job should be inserted based on
+        the given timestamp.
+
+        :type timestamp: int
+        :type job_id: str
+        """
+
+        lo, hi = 0, len(self._jobs)
+        timestamp = float('inf') if timestamp is None else timestamp
+        while lo < hi:
+            mid = (lo + hi) // 2
+            mid_job, mid_timestamp = self._jobs[mid]
+            mid_timestamp = float('inf') if mid_timestamp is None else mid_timestamp
+            if mid_timestamp > timestamp:
+                hi = mid
+            elif mid_timestamp < timestamp:
+                lo = mid + 1
+            elif mid_job.id > job_id:
+                hi = mid
+            elif mid_job.id < job_id:
+                lo = mid + 1
+            else:
+                return mid
+
+        return lo
@@ -0,0 +1,124 @@
+from __future__ import absolute_import
+
+from apscheduler.jobstores.base import BaseJobStore, JobLookupError, ConflictingIdError
+from apscheduler.util import maybe_ref, datetime_to_utc_timestamp, utc_timestamp_to_datetime
+from apscheduler.job import Job
+
+try:
+    import cPickle as pickle
+except ImportError:  # pragma: nocover
+    import pickle
+
+try:
+    from bson.binary import Binary
+    from pymongo.errors import DuplicateKeyError
+    from pymongo import MongoClient, ASCENDING
+except ImportError:  # pragma: nocover
+    raise ImportError('MongoDBJobStore requires PyMongo installed')
+
+
+class MongoDBJobStore(BaseJobStore):
+    """
+    Stores jobs in a MongoDB database. Any leftover keyword arguments are directly passed to pymongo's `MongoClient
+    <http://api.mongodb.org/python/current/api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient>`_.
+
+    Plugin alias: ``mongodb``
+
+    :param str database: database to store jobs in
+    :param str collection: collection to store jobs in
+    :param client: a :class:`~pymongo.mongo_client.MongoClient` instance to use instead of providing connection
+                   arguments
+    :param int pickle_protocol: pickle protocol level to use (for serialization), defaults to the highest available
+    """
+
+    def __init__(self, database='apscheduler', collection='jobs', client=None,
+                 pickle_protocol=pickle.HIGHEST_PROTOCOL, **connect_args):
+        super(MongoDBJobStore, self).__init__()
+        self.pickle_protocol = pickle_protocol
+
+        if not database:
+            raise ValueError('The "database" parameter must not be empty')
+        if not collection:
+            raise ValueError('The "collection" parameter must not be empty')
+
+        if client:
+            self.connection = maybe_ref(client)
+        else:
+            connect_args.setdefault('w', 1)
+            self.connection = MongoClient(**connect_args)
+
+        self.collection = self.connection[database][collection]
+        self.collection.ensure_index('next_run_time', sparse=True)
+
+    def lookup_job(self, job_id):
+        document = self.collection.find_one(job_id, ['job_state'])
+        return self._reconstitute_job(document['job_state']) if document else None
+
+    def get_due_jobs(self, now):
+        timestamp = datetime_to_utc_timestamp(now)
+        return self._get_jobs({'next_run_time': {'$lte': timestamp}})
+
+    def get_next_run_time(self):
+        document = self.collection.find_one({'next_run_time': {'$ne': None}}, fields=['next_run_time'],
+                                            sort=[('next_run_time', ASCENDING)])
+        return utc_timestamp_to_datetime(document['next_run_time']) if document else None
+
+    def get_all_jobs(self):
+        return self._get_jobs({})
+
+    def add_job(self, job):
+        try:
+            self.collection.insert({
+                '_id': job.id,
+                'next_run_time': datetime_to_utc_timestamp(job.next_run_time),
+                'job_state': Binary(pickle.dumps(job.__getstate__(), self.pickle_protocol))
+            })
+        except DuplicateKeyError:
+            raise ConflictingIdError(job.id)
+
+    def update_job(self, job):
+        changes = {
+            'next_run_time': datetime_to_utc_timestamp(job.next_run_time),
+            'job_state': Binary(pickle.dumps(job.__getstate__(), self.pickle_protocol))
+        }
+        result = self.collection.update({'_id': job.id}, {'$set': changes})
+        if result and result['n'] == 0:
+            raise JobLookupError(id)
+
+    def remove_job(self, job_id):
+        result = self.collection.remove(job_id)
+        if result and result['n'] == 0:
+            raise JobLookupError(job_id)
+
+    def remove_all_jobs(self):
+        self.collection.remove()
+
+    def shutdown(self):
+        self.connection.disconnect()
+
+    def _reconstitute_job(self, job_state):
+        job_state = pickle.loads(job_state)
+        job = Job.__new__(Job)
+        job.__setstate__(job_state)
+        job._scheduler = self._scheduler
+        job._jobstore_alias = self._alias
+        return job
+
+    def _get_jobs(self, conditions):
+        jobs = []
+        failed_job_ids = []
+        for document in self.collection.find(conditions, ['_id', 'job_state'], sort=[('next_run_time', ASCENDING)]):
+            try:
+                jobs.append(self._reconstitute_job(document['job_state']))
+            except:
+                self._logger.exception('Unable to restore job "%s" -- removing it', document['_id'])
+                failed_job_ids.append(document['_id'])
+
+        # Remove all the jobs we failed to restore
+        if failed_job_ids:
+            self.collection.remove({'_id': {'$in': failed_job_ids}})
+
+        return jobs
+
+    def __repr__(self):
+        return '<%s (client=%s)>' % (self.__class__.__name__, self.connection)
@@ -1,84 +0,0 @@
-"""
-Stores jobs in a MongoDB database.
-"""
-import logging
-
-from apscheduler.jobstores.base import JobStore
-from apscheduler.job import Job
-
-try:
-    import cPickle as pickle
-except ImportError:  # pragma: nocover
-    import pickle
-
-try:
-    from bson.binary import Binary
-    from pymongo.connection import Connection
-except ImportError:  # pragma: nocover
-    raise ImportError('MongoDBJobStore requires PyMongo installed')
-
-logger = logging.getLogger(__name__)
-
-
-class MongoDBJobStore(JobStore):
-    def __init__(self, database='apscheduler', collection='jobs',
-                 connection=None, pickle_protocol=pickle.HIGHEST_PROTOCOL,
-                 **connect_args):
-        self.jobs = []
-        self.pickle_protocol = pickle_protocol
-
-        if not database:
-            raise ValueError('The "database" parameter must not be empty')
-        if not collection:
-            raise ValueError('The "collection" parameter must not be empty')
-
-        if connection:
-            self.connection = connection
-        else:
-            self.connection = Connection(**connect_args)
-
-        self.collection = self.connection[database][collection]
-
-    def add_job(self, job):
-        job_dict = job.__getstate__()
-        job_dict['trigger'] = Binary(pickle.dumps(job.trigger,
-                                                  self.pickle_protocol))
-        job_dict['args'] = Binary(pickle.dumps(job.args,
-                                               self.pickle_protocol))
-        job_dict['kwargs'] = Binary(pickle.dumps(job.kwargs,
-                                                 self.pickle_protocol))
-        job.id = self.collection.insert(job_dict)
-        self.jobs.append(job)
-
-    def remove_job(self, job):
-        self.collection.remove(job.id)
-        self.jobs.remove(job)
-
-    def load_jobs(self):
-        jobs = []
-        for job_dict in self.collection.find():
-            try:
-                job = Job.__new__(Job)
-                job_dict['id'] = job_dict.pop('_id')
-                job_dict['trigger'] = pickle.loads(job_dict['trigger'])
-                job_dict['args'] = pickle.loads(job_dict['args'])
-                job_dict['kwargs'] = pickle.loads(job_dict['kwargs'])
-                job.__setstate__(job_dict)
-                jobs.append(job)
-            except Exception:
-                job_name = job_dict.get('name', '(unknown)')
-                logger.exception('Unable to restore job "%s"', job_name)
-        self.jobs = jobs
-
-    def update_job(self, job):
-        spec = {'_id': job.id}
-        document = {'$set': {'next_run_time': job.next_run_time},
-                    '$inc': {'runs': 1}}
-        self.collection.update(spec, document)
-
-    def close(self):
-        self.connection.disconnect()
-
-    def __repr__(self):
-        connection = self.collection.database.connection
-        return '<%s (connection=%s)>' % (self.__class__.__name__, connection)
@@ -1,25 +0,0 @@
-"""
-Stores jobs in an array in RAM. Provides no persistence support.
-"""
-
-from apscheduler.jobstores.base import JobStore
-
-
-class RAMJobStore(JobStore):
-    def __init__(self):
-        self.jobs = []
-
-    def add_job(self, job):
-        self.jobs.append(job)
-
-    def update_job(self, job):
-        pass
-
-    def remove_job(self, job):
-        self.jobs.remove(job)
-
-    def load_jobs(self):
-        pass
-
-    def __repr__(self):
-        return '<%s>' % (self.__class__.__name__)
@@ -0,0 +1,138 @@
+from __future__ import absolute_import
+
+import six
+
+from apscheduler.jobstores.base import BaseJobStore, JobLookupError, ConflictingIdError
+from apscheduler.util import datetime_to_utc_timestamp, utc_timestamp_to_datetime
+from apscheduler.job import Job
+
+try:
+    import cPickle as pickle
+except ImportError:  # pragma: nocover
+    import pickle
+
+try:
+    from redis import StrictRedis
+except ImportError:  # pragma: nocover
+    raise ImportError('RedisJobStore requires redis installed')
+
+
+class RedisJobStore(BaseJobStore):
+    """
+    Stores jobs in a Redis database. Any leftover keyword arguments are directly passed to redis's StrictRedis.
+
+    Plugin alias: ``redis``
+
+    :param int db: the database number to store jobs in
+    :param str jobs_key: key to store jobs in
+    :param str run_times_key: key to store the jobs' run times in
+    :param int pickle_protocol: pickle protocol level to use (for serialization), defaults to the highest available
+    """
+
+    def __init__(self, db=0, jobs_key='apscheduler.jobs', run_times_key='apscheduler.run_times',
+                 pickle_protocol=pickle.HIGHEST_PROTOCOL, **connect_args):
+        super(RedisJobStore, self).__init__()
+
+        if db is None:
+            raise ValueError('The "db" parameter must not be empty')
+        if not jobs_key:
+            raise ValueError('The "jobs_key" parameter must not be empty')
+        if not run_times_key:
+            raise ValueError('The "run_times_key" parameter must not be empty')
+
+        self.pickle_protocol = pickle_protocol
+        self.jobs_key = jobs_key
+        self.run_times_key = run_times_key
+        self.redis = StrictRedis(db=int(db), **connect_args)
+
+    def lookup_job(self, job_id):
+        job_state = self.redis.hget(self.jobs_key, job_id)
+        return self._reconstitute_job(job_state) if job_state else None
+
+    def get_due_jobs(self, now):
+        timestamp = datetime_to_utc_timestamp(now)
+        job_ids = self.redis.zrangebyscore(self.run_times_key, 0, timestamp)
+        if job_ids:
+            job_states = self.redis.hmget(self.jobs_key, *job_ids)
+            return self._reconstitute_jobs(six.moves.zip(job_ids, job_states))
+        return []
+
+    def get_next_run_time(self):
+        next_run_time = self.redis.zrange(self.run_times_key, 0, 0, withscores=True)
+        if next_run_time:
+            return utc_timestamp_to_datetime(next_run_time[0][1])
+
+    def get_all_jobs(self):
+        job_states = self.redis.hgetall(self.jobs_key)
+        jobs = self._reconstitute_jobs(six.iteritems(job_states))
+        return sorted(jobs, key=lambda job: job.next_run_time)
+
+    def add_job(self, job):
+        if self.redis.hexists(self.jobs_key, job.id):
+            raise ConflictingIdError(job.id)
+
+        with self.redis.pipeline() as pipe:
+            pipe.multi()
+            pipe.hset(self.jobs_key, job.id, pickle.dumps(job.__getstate__(), self.pickle_protocol))
+            pipe.zadd(self.run_times_key, datetime_to_utc_timestamp(job.next_run_time), job.id)
+            pipe.execute()
+
+    def update_job(self, job):
+        if not self.redis.hexists(self.jobs_key, job.id):
+            raise JobLookupError(job.id)
+
+        with self.redis.pipeline() as pipe:
+            pipe.hset(self.jobs_key, job.id, pickle.dumps(job.__getstate__(), self.pickle_protocol))
+            if job.next_run_time:
+                pipe.zadd(self.run_times_key, datetime_to_utc_timestamp(job.next_run_time), job.id)
+            else:
+                pipe.zrem(self.run_times_key, job.id)
+            pipe.execute()
+
+    def remove_job(self, job_id):
+        if not self.redis.hexists(self.jobs_key, job_id):
+            raise JobLookupError(job_id)
+
+        with self.redis.pipeline() as pipe:
+            pipe.hdel(self.jobs_key, job_id)
+            pipe.zrem(self.run_times_key, job_id)
+            pipe.execute()
+
+    def remove_all_jobs(self):
+        with self.redis.pipeline() as pipe:
+            pipe.delete(self.jobs_key)
+            pipe.delete(self.run_times_key)
+            pipe.execute()
+
+    def shutdown(self):
+        self.redis.connection_pool.disconnect()
+
+    def _reconstitute_job(self, job_state):
+        job_state = pickle.loads(job_state)
+        job = Job.__new__(Job)
+        job.__setstate__(job_state)
+        job._scheduler = self._scheduler
+        job._jobstore_alias = self._alias
+        return job
+
+    def _reconstitute_jobs(self, job_states):
+        jobs = []
+        failed_job_ids = []
+        for job_id, job_state in job_states:
+            try:
+                jobs.append(self._reconstitute_job(job_state))
+            except:
+                self._logger.exception('Unable to restore job "%s" -- removing it', job_id)
+                failed_job_ids.append(job_id)
+
+        # Remove all the jobs we failed to restore
+        if failed_job_ids:
+            with self.redis.pipeline() as pipe:
+                pipe.hdel(self.jobs_key, *failed_job_ids)
+                pipe.zrem(self.run_times_key, *failed_job_ids)
+                pipe.execute()
+
+        return jobs
+
+    def __repr__(self):
+        return '<%s>' % self.__class__.__name__
@@ -1,65 +0,0 @@
-"""
-Stores jobs in a file governed by the :mod:`shelve` module.
-"""
-
-import shelve
-import pickle
-import random
-import logging
-
-from apscheduler.jobstores.base import JobStore
-from apscheduler.job import Job
-from apscheduler.util import itervalues
-
-logger = logging.getLogger(__name__)
-
-
-class ShelveJobStore(JobStore):
-    MAX_ID = 1000000
-
-    def __init__(self, path, pickle_protocol=pickle.HIGHEST_PROTOCOL):
-        self.jobs = []
-        self.path = path
-        self.pickle_protocol = pickle_protocol
-        self.store = shelve.open(path, 'c', self.pickle_protocol)
-
-    def _generate_id(self):
-        id = None
-        while not id:
-            id = str(random.randint(1, self.MAX_ID))
-            if not id in self.store:
-                return id
-
-    def add_job(self, job):
-        job.id = self._generate_id()
-        self.jobs.append(job)
-        self.store[job.id] = job.__getstate__()
-
-    def update_job(self, job):
-        job_dict = self.store[job.id]
-        job_dict['next_run_time'] = job.next_run_time
-        job_dict['runs'] = job.runs
-        self.store[job.id] = job_dict
-
-    def remove_job(self, job):
-        del self.store[job.id]
-        self.jobs.remove(job)
-
-    def load_jobs(self):
-        jobs = []
-        for job_dict in itervalues(self.store):
-            try:
-                job = Job.__new__(Job)
-                job.__setstate__(job_dict)
-                jobs.append(job)
-            except Exception:
-                job_name = job_dict.get('name', '(unknown)')
-                logger.exception('Unable to restore job "%s"', job_name)
-
-        self.jobs = jobs
-
-    def close(self):
-        self.store.close()
-
-    def __repr__(self):
-        return '<%s (path=%s)>' % (self.__class__.__name__, self.path)
@@ -0,0 +1,137 @@
+from __future__ import absolute_import
+
+from apscheduler.jobstores.base import BaseJobStore, JobLookupError, ConflictingIdError
+from apscheduler.util import maybe_ref, datetime_to_utc_timestamp, utc_timestamp_to_datetime
+from apscheduler.job import Job
+
+try:
+    import cPickle as pickle
+except ImportError:  # pragma: nocover
+    import pickle
+
+try:
+    from sqlalchemy import create_engine, Table, Column, MetaData, Unicode, Float, LargeBinary, select
+    from sqlalchemy.exc import IntegrityError
+except ImportError:  # pragma: nocover
+    raise ImportError('SQLAlchemyJobStore requires SQLAlchemy installed')
+
+
+class SQLAlchemyJobStore(BaseJobStore):
+    """
+    Stores jobs in a database table using SQLAlchemy. The table will be created if it doesn't exist in the database.
+
+    Plugin alias: ``sqlalchemy``
+
+    :param str url: connection string (see `SQLAlchemy documentation
+                    <http://docs.sqlalchemy.org/en/latest/core/engines.html?highlight=create_engine#database-urls>`_
+                    on this)
+    :param engine: an SQLAlchemy Engine to use instead of creating a new one based on ``url``
+    :param str tablename: name of the table to store jobs in
+    :param metadata: a :class:`~sqlalchemy.MetaData` instance to use instead of creating a new one
+    :param int pickle_protocol: pickle protocol level to use (for serialization), defaults to the highest available
+    """
+
+    def __init__(self, url=None, engine=None, tablename='apscheduler_jobs', metadata=None,
+                 pickle_protocol=pickle.HIGHEST_PROTOCOL):
+        super(SQLAlchemyJobStore, self).__init__()
+        self.pickle_protocol = pickle_protocol
+        metadata = maybe_ref(metadata) or MetaData()
+
+        if engine:
+            self.engine = maybe_ref(engine)
+        elif url:
+            self.engine = create_engine(url)
+        else:
+            raise ValueError('Need either "engine" or "url" defined')
+
+        # 191 = max key length in MySQL for InnoDB/utf8mb4 tables, 25 = precision that translates to an 8-byte float
+        self.jobs_t = Table(
+            tablename, metadata,
+            Column('id', Unicode(191, _warn_on_bytestring=False), primary_key=True),
+            Column('next_run_time', Float(25), index=True),
+            Column('job_state', LargeBinary, nullable=False)
+        )
+
+        self.jobs_t.create(self.engine, True)
+
+    def lookup_job(self, job_id):
+        selectable = select([self.jobs_t.c.job_state]).where(self.jobs_t.c.id == job_id)
+        job_state = self.engine.execute(selectable).scalar()
+        return self._reconstitute_job(job_state) if job_state else None
+
+    def get_due_jobs(self, now):
+        timestamp = datetime_to_utc_timestamp(now)
+        return self._get_jobs(self.jobs_t.c.next_run_time <= timestamp)
+
+    def get_next_run_time(self):
+        selectable = select([self.jobs_t.c.next_run_time]).where(self.jobs_t.c.next_run_time != None).\
+            order_by(self.jobs_t.c.next_run_time).limit(1)
+        next_run_time = self.engine.execute(selectable).scalar()
+        return utc_timestamp_to_datetime(next_run_time)
+
+    def get_all_jobs(self):
+        return self._get_jobs()
+
+    def add_job(self, job):
+        insert = self.jobs_t.insert().values(**{
+            'id': job.id,
+            'next_run_time': datetime_to_utc_timestamp(job.next_run_time),
+            'job_state': pickle.dumps(job.__getstate__(), self.pickle_protocol)
+        })
+        try:
+            self.engine.execute(insert)
+        except IntegrityError:
+            raise ConflictingIdError(job.id)
+
+    def update_job(self, job):
+        update = self.jobs_t.update().values(**{
+            'next_run_time': datetime_to_utc_timestamp(job.next_run_time),
+            'job_state': pickle.dumps(job.__getstate__(), self.pickle_protocol)
+        }).where(self.jobs_t.c.id == job.id)
+        result = self.engine.execute(update)
+        if result.rowcount == 0:
+            raise JobLookupError(id)
+
+    def remove_job(self, job_id):
+        delete = self.jobs_t.delete().where(self.jobs_t.c.id == job_id)
+        result = self.engine.execute(delete)
+        if result.rowcount == 0:
+            raise JobLookupError(job_id)
+
+    def remove_all_jobs(self):
+        delete = self.jobs_t.delete()
+        self.engine.execute(delete)
+
+    def shutdown(self):
+        self.engine.dispose()
+
+    def _reconstitute_job(self, job_state):
+        job_state = pickle.loads(job_state)
+        job_state['jobstore'] = self
+        job = Job.__new__(Job)
+        job.__setstate__(job_state)
+        job._scheduler = self._scheduler
+        job._jobstore_alias = self._alias
+        return job
+
+    def _get_jobs(self, *conditions):
+        jobs = []
+        selectable = select([self.jobs_t.c.id, self.jobs_t.c.job_state]).order_by(self.jobs_t.c.next_run_time)
+        selectable = selectable.where(*conditions) if conditions else selectable
+        failed_job_ids = set()
+        for row in self.engine.execute(selectable):
+            try:
+                jobs.append(self._reconstitute_job(row.job_state))
+            except:
+                self._logger.exception('Unable to restore job "%s" -- removing it', row.id)
+                failed_job_ids.add(row.id)
+
+        # Remove all the jobs we failed to restore
+        if failed_job_ids:
+            delete = self.jobs_t.delete().where(self.jobs_t.c.id.in_(failed_job_ids))
+            self.engine.execute(delete)
+
+        return jobs
+
+    def __repr__(self):
+        return '<%s (url=%s)>' % (self.__class__.__name__, self.engine.url)
@@ -1,87 +0,0 @@
-"""
-Stores jobs in a database table using SQLAlchemy.
-"""
-import pickle
-import logging
-
-from apscheduler.jobstores.base import JobStore
-from apscheduler.job import Job
-
-try:
-    from sqlalchemy import *
-except ImportError:  # pragma: nocover
-    raise ImportError('SQLAlchemyJobStore requires SQLAlchemy installed')
-
-logger = logging.getLogger(__name__)
-
-
-class SQLAlchemyJobStore(JobStore):
-    def __init__(self, url=None, engine=None, tablename='apscheduler_jobs',
-                 metadata=None, pickle_protocol=pickle.HIGHEST_PROTOCOL):
-        self.jobs = []
-        self.pickle_protocol = pickle_protocol
-
-        if engine:
-            self.engine = engine
-        elif url:
-            self.engine = create_engine(url)
-        else:
-            raise ValueError('Need either "engine" or "url" defined')
-
-        self.jobs_t = Table(tablename, metadata or MetaData(),
-            Column('id', Integer,
-                   Sequence(tablename + '_id_seq', optional=True),
-                   primary_key=True),
-            Column('trigger', PickleType(pickle_protocol, mutable=False),
-                   nullable=False),
-            Column('func_ref', String(1024), nullable=False),
-            Column('args', PickleType(pickle_protocol, mutable=False),
-                   nullable=False),
-            Column('kwargs', PickleType(pickle_protocol, mutable=False),
-                   nullable=False),
-            Column('name', Unicode(1024), unique=True),
-            Column('misfire_grace_time', Integer, nullable=False),
-            Column('coalesce', Boolean, nullable=False),
-            Column('max_runs', Integer),
-            Column('max_instances', Integer),
-            Column('next_run_time', DateTime, nullable=False),
-            Column('runs', BigInteger))
-
-        self.jobs_t.create(self.engine, True)
-
-    def add_job(self, job):
-        job_dict = job.__getstate__()
-        result = self.engine.execute(self.jobs_t.insert().values(**job_dict))
-        job.id = result.inserted_primary_key[0]
-        self.jobs.append(job)
-
-    def remove_job(self, job):
-        delete = self.jobs_t.delete().where(self.jobs_t.c.id == job.id)
-        self.engine.execute(delete)
-        self.jobs.remove(job)
-
-    def load_jobs(self):
-        jobs = []
-        for row in self.engine.execute(select([self.jobs_t])):
-            try:
-                job = Job.__new__(Job)
-                job_dict = dict(row.items())
-                job.__setstate__(job_dict)
-                jobs.append(job)
-            except Exception:
-                job_name = job_dict.get('name', '(unknown)')
-                logger.exception('Unable to restore job "%s"', job_name)
-        self.jobs = jobs
-
-    def update_job(self, job):
-        job_dict = job.__getstate__()
-        update = self.jobs_t.update().where(self.jobs_t.c.id == job.id).\
-            values(next_run_time=job_dict['next_run_time'],
-                   runs=job_dict['runs'])
-        self.engine.execute(update)
-
-    def close(self):
-        self.engine.dispose()
-
-    def __repr__(self):
-        return '<%s (url=%s)>' % (self.__class__.__name__, self.engine.url)
@@ -1,559 +0,0 @@
-"""
-This module is the main part of the library. It houses the Scheduler class
-and related exceptions.
-"""
-
-from threading import Thread, Event, Lock
-from datetime import datetime, timedelta
-from logging import getLogger
-import os
-import sys
-
-from apscheduler.util import *
-from apscheduler.triggers import SimpleTrigger, IntervalTrigger, CronTrigger
-from apscheduler.jobstores.ram_store import RAMJobStore
-from apscheduler.job import Job, MaxInstancesReachedError
-from apscheduler.events import *
-from apscheduler.threadpool import ThreadPool
-
-logger = getLogger(__name__)
-
-
-class SchedulerAlreadyRunningError(Exception):
-    """
-    Raised when attempting to start or configure the scheduler when it's
-    already running.
-    """
-
-    def __str__(self):
-        return 'Scheduler is already running'
-
-
-class Scheduler(object):
-    """
-    This class is responsible for scheduling jobs and triggering
-    their execution.
-    """
-
-    _stopped = False
-    _thread = None
-
-    def __init__(self, gconfig={}, **options):
-        self._wakeup = Event()
-        self._jobstores = {}
-        self._jobstores_lock = Lock()
-        self._listeners = []
-        self._listeners_lock = Lock()
-        self._pending_jobs = []
-        self.configure(gconfig, **options)
-
-    def configure(self, gconfig={}, **options):
-        """
-        Reconfigures the scheduler with the given options. Can only be done
-        when the scheduler isn't running.
-        """
-        if self.running:
-            raise SchedulerAlreadyRunningError
-
-        # Set general options
-        config = combine_opts(gconfig, 'apscheduler.', options)
-        self.misfire_grace_time = int(config.pop('misfire_grace_time', 1))
-        self.coalesce = asbool(config.pop('coalesce', True))
-        self.daemonic = asbool(config.pop('daemonic', True))
-
-        # Configure the thread pool
-        if 'threadpool' in config:
-            self._threadpool = maybe_ref(config['threadpool'])
-        else:
-            threadpool_opts = combine_opts(config, 'threadpool.')
-            self._threadpool = ThreadPool(**threadpool_opts)
-
-        # Configure job stores
-        jobstore_opts = combine_opts(config, 'jobstore.')
-        jobstores = {}
-        for key, value in jobstore_opts.items():
-            store_name, option = key.split('.', 1)
-            opts_dict = jobstores.setdefault(store_name, {})
-            opts_dict[option] = value
-
-        for alias, opts in jobstores.items():
-            classname = opts.pop('class')
-            cls = maybe_ref(classname)
-            jobstore = cls(**opts)
-            self.add_jobstore(jobstore, alias, True)
-
-    def start(self):
-        """
-        Starts the scheduler in a new thread.
-        """
-        if self.running:
-            raise SchedulerAlreadyRunningError
-
-        # Create a RAMJobStore as the default if there is no default job store
-        if not 'default' in self._jobstores:
-            self.add_jobstore(RAMJobStore(), 'default', True)
-
-        # Schedule all pending jobs
-        for job, jobstore in self._pending_jobs:
-            self._real_add_job(job, jobstore, False)
-        del self._pending_jobs[:]
-
-        self._stopped = False
-        self._thread = Thread(target=self._main_loop, name='APScheduler')
-        self._thread.setDaemon(self.daemonic)
-        self._thread.start()
-
-    def shutdown(self, wait=True, shutdown_threadpool=True):
-        """
-        Shuts down the scheduler and terminates the thread.
-        Does not interrupt any currently running jobs.
-
-        :param wait: ``True`` to wait until all currently executing jobs have
-                     finished (if ``shutdown_threadpool`` is also ``True``)
-        :param shutdown_threadpool: ``True`` to shut down the thread pool
-        """
-        if not self.running:
-            return
-
-        self._stopped = True
-        self._wakeup.set()
-
-        # Shut down the thread pool
-        if shutdown_threadpool:
-            self._threadpool.shutdown(wait)
-
-        # Wait until the scheduler thread terminates
-        self._thread.join()
-
-    @property
-    def running(self):
-        return not self._stopped and self._thread and self._thread.isAlive()
-
-    def add_jobstore(self, jobstore, alias, quiet=False):
-        """
-        Adds a job store to this scheduler.
-
-        :param jobstore: job store to be added
-        :param alias: alias for the job store
-        :param quiet: True to suppress scheduler thread wakeup
-        :type jobstore: instance of
-            :class:`~apscheduler.jobstores.base.JobStore`
-        :type alias: str
-        """
-        self._jobstores_lock.acquire()
-        try:
-            if alias in self._jobstores:
-                raise KeyError('Alias "%s" is already in use' % alias)
-            self._jobstores[alias] = jobstore
-            jobstore.load_jobs()
-        finally:
-            self._jobstores_lock.release()
-
-        # Notify listeners that a new job store has been added
-        self._notify_listeners(JobStoreEvent(EVENT_JOBSTORE_ADDED, alias))
-
-        # Notify the scheduler so it can scan the new job store for jobs
-        if not quiet:
-            self._wakeup.set()
-
-    def remove_jobstore(self, alias):
-        """
-        Removes the job store by the given alias from this scheduler.
-
-        :type alias: str
-        """
-        self._jobstores_lock.acquire()
-        try:
-            try:
-                del self._jobstores[alias]
-            except KeyError:
-                raise KeyError('No such job store: %s' % alias)
-        finally:
-            self._jobstores_lock.release()
-
-        # Notify listeners that a job store has been removed
-        self._notify_listeners(JobStoreEvent(EVENT_JOBSTORE_REMOVED, alias))
-
-    def add_listener(self, callback, mask=EVENT_ALL):
-        """
-        Adds a listener for scheduler events. When a matching event occurs,
-        ``callback`` is executed with the event object as its sole argument.
-        If the ``mask`` parameter is not provided, the callback will receive
-        events of all types.
-
-        :param callback: any callable that takes one argument
-        :param mask: bitmask that indicates which events should be listened to
-        """
-        self._listeners_lock.acquire()
-        try:
-            self._listeners.append((callback, mask))
-        finally:
-            self._listeners_lock.release()
-
-    def remove_listener(self, callback):
-        """
-        Removes a previously added event listener.
-        """
-        self._listeners_lock.acquire()
-        try:
-            for i, (cb, _) in enumerate(self._listeners):
-                if callback == cb:
-                    del self._listeners[i]
-        finally:
-            self._listeners_lock.release()
-
-    def _notify_listeners(self, event):
-        self._listeners_lock.acquire()
-        try:
-            listeners = tuple(self._listeners)
-        finally:
-            self._listeners_lock.release()
-
-        for cb, mask in listeners:
-            if event.code & mask:
-                try:
-                    cb(event)
-                except:
-                    logger.exception('Error notifying listener')
-
-    def _real_add_job(self, job, jobstore, wakeup):
-        job.compute_next_run_time(datetime.now())
-        if not job.next_run_time:
-            raise ValueError('Not adding job since it would never be run')
-
-        self._jobstores_lock.acquire()
-        try:
-            try:
-                store = self._jobstores[jobstore]
-            except KeyError:
-                raise KeyError('No such job store: %s' % jobstore)
-            store.add_job(job)
-        finally:
-            self._jobstores_lock.release()
-
-        # Notify listeners that a new job has been added
-        event = JobStoreEvent(EVENT_JOBSTORE_JOB_ADDED, jobstore, job)
-        self._notify_listeners(event)
-
-        logger.info('Added job "%s" to job store "%s"', job, jobstore)
-
-        # Notify the scheduler about the new job
-        if wakeup:
-            self._wakeup.set()
-
-    def add_job(self, trigger, func, args, kwargs, jobstore='default',
-                **options):
-        """
-        Adds the given job to the job list and notifies the scheduler thread.
-
-        :param trigger: alias of the job store to store the job in
-        :param func: callable to run at the given time
-        :param args: list of positional arguments to call func with
-        :param kwargs: dict of keyword arguments to call func with
-        :param jobstore: alias of the job store to store the job in
-        :rtype: :class:`~apscheduler.job.Job`
-        """
-        job = Job(trigger, func, args or [], kwargs or {},
-                  options.pop('misfire_grace_time', self.misfire_grace_time),
-                  options.pop('coalesce', self.coalesce), **options)
-        if not self.running:
-            self._pending_jobs.append((job, jobstore))
-            logger.info('Adding job tentatively -- it will be properly '
-                        'scheduled when the scheduler starts')
-        else:
-            self._real_add_job(job, jobstore, True)
-        return job
-
-    def _remove_job(self, job, alias, jobstore):
-        jobstore.remove_job(job)
-
-        # Notify listeners that a job has been removed
-        event = JobStoreEvent(EVENT_JOBSTORE_JOB_REMOVED, alias, job)
-        self._notify_listeners(event)
-
-        logger.info('Removed job "%s"', job)
-
-    def add_date_job(self, func, date, args=None, kwargs=None, **options):
-        """
-        Schedules a job to be completed on a specific date and time.
-
-        :param func: callable to run at the given time
-        :param date: the date/time to run the job at
-        :param name: name of the job
-        :param jobstore: stored the job in the named (or given) job store
-        :param misfire_grace_time: seconds after the designated run time that
-            the job is still allowed to be run
-        :type date: :class:`datetime.date`
-        :rtype: :class:`~apscheduler.job.Job`
-        """
-        trigger = SimpleTrigger(date)
-        return self.add_job(trigger, func, args, kwargs, **options)
-
-    def add_interval_job(self, func, weeks=0, days=0, hours=0, minutes=0,
-                         seconds=0, start_date=None, args=None, kwargs=None,
-                         **options):
-        """
-        Schedules a job to be completed on specified intervals.
-
-        :param func: callable to run
-        :param weeks: number of weeks to wait
-        :param days: number of days to wait
-        :param hours: number of hours to wait
-        :param minutes: number of minutes to wait
-        :param seconds: number of seconds to wait
-        :param start_date: when to first execute the job and start the
-            counter (default is after the given interval)
-        :param args: list of positional arguments to call func with
-        :param kwargs: dict of keyword arguments to call func with
-        :param name: name of the job
-        :param jobstore: alias of the job store to add the job to
-        :param misfire_grace_time: seconds after the designated run time that
-            the job is still allowed to be run
-        :rtype: :class:`~apscheduler.job.Job`
-        """
-        interval = timedelta(weeks=weeks, days=days, hours=hours,
-                             minutes=minutes, seconds=seconds)
-        trigger = IntervalTrigger(interval, start_date)
-        return self.add_job(trigger, func, args, kwargs, **options)
-
-    def add_cron_job(self, func, year='*', month='*', day='*', week='*',
-                     day_of_week='*', hour='*', minute='*', second='*',
-                     start_date=None, args=None, kwargs=None, **options):
-        """
-        Schedules a job to be completed on times that match the given
-        expressions.
-
-        :param func: callable to run
-        :param year: year to run on
-        :param month: month to run on (0 = January)
-        :param day: day of month to run on
-        :param week: week of the year to run on
-        :param day_of_week: weekday to run on (0 = Monday)
-        :param hour: hour to run on
-        :param second: second to run on
-        :param args: list of positional arguments to call func with
-        :param kwargs: dict of keyword arguments to call func with
-        :param name: name of the job
-        :param jobstore: alias of the job store to add the job to
-        :param misfire_grace_time: seconds after the designated run time that
-            the job is still allowed to be run
-        :return: the scheduled job
-        :rtype: :class:`~apscheduler.job.Job`
-        """
-        trigger = CronTrigger(year=year, month=month, day=day, week=week,
-                              day_of_week=day_of_week, hour=hour,
-                              minute=minute, second=second,
-                              start_date=start_date)
-        return self.add_job(trigger, func, args, kwargs, **options)
-
-    def cron_schedule(self, **options):
-        """
-        Decorator version of :meth:`add_cron_job`.
-        This decorator does not wrap its host function.
-        Unscheduling decorated functions is possible by passing the ``job``
-        attribute of the scheduled function to :meth:`unschedule_job`.
-        """
-        def inner(func):
-            func.job = self.add_cron_job(func, **options)
-            return func
-        return inner
-
-    def interval_schedule(self, **options):
-        """
-        Decorator version of :meth:`add_interval_job`.
-        This decorator does not wrap its host function.
-        Unscheduling decorated functions is possible by passing the ``job``
-        attribute of the scheduled function to :meth:`unschedule_job`.
-        """
-        def inner(func):
-            func.job = self.add_interval_job(func, **options)
-            return func
-        return inner
-
-    def get_jobs(self):
-        """
-        Returns a list of all scheduled jobs.
-
-        :return: list of :class:`~apscheduler.job.Job` objects
-        """
-        self._jobstores_lock.acquire()
-        try:
-            jobs = []
-            for jobstore in itervalues(self._jobstores):
-                jobs.extend(jobstore.jobs)
-            return jobs
-        finally:
-            self._jobstores_lock.release()
-
-    def unschedule_job(self, job):
-        """
-        Removes a job, preventing it from being run any more.
-        """
-        self._jobstores_lock.acquire()
-        try:
-            for alias, jobstore in iteritems(self._jobstores):
-                if job in list(jobstore.jobs):
-                    self._remove_job(job, alias, jobstore)
-                    return
-        finally:
-            self._jobstores_lock.release()
-
-        raise KeyError('Job "%s" is not scheduled in any job store' % job)
-
-    def unschedule_func(self, func):
-        """
-        Removes all jobs that would execute the given function.
-        """
-        found = False
-        self._jobstores_lock.acquire()
-        try:
-            for alias, jobstore in iteritems(self._jobstores):
-                for job in list(jobstore.jobs):
-                    if job.func == func:
-                        self._remove_job(job, alias, jobstore)
-                        found = True
-        finally:
-            self._jobstores_lock.release()
-
-        if not found:
-            raise KeyError('The given function is not scheduled in this '
-                           'scheduler')
-
-    def print_jobs(self, out=None):
-        """
-        Prints out a textual listing of all jobs currently scheduled on this
-        scheduler.
-
-        :param out: a file-like object to print to (defaults to **sys.stdout**
-                    if nothing is given)
-        """
-        out = out or sys.stdout
-        job_strs = []
-        self._jobstores_lock.acquire()
-        try:
-            for alias, jobstore in iteritems(self._jobstores):
-                job_strs.append('Jobstore %s:' % alias)
-                if jobstore.jobs:
-                    for job in jobstore.jobs:
-                        job_strs.append('    %s' % job)
-                else:
-                    job_strs.append('    No scheduled jobs')
-        finally:
-            self._jobstores_lock.release()
-
-        out.write(os.linesep.join(job_strs))
-
-    def _run_job(self, job, run_times):
-        """
-        Acts as a harness that runs the actual job code in a thread.
-        """
-        for run_time in run_times:
-            # See if the job missed its run time window, and handle possible
-            # misfires accordingly
-            difference = datetime.now() - run_time
-            grace_time = timedelta(seconds=job.misfire_grace_time)
-            if difference > grace_time:
-                # Notify listeners about a missed run
-                event = JobEvent(EVENT_JOB_MISSED, job, run_time)
-                self._notify_listeners(event)
-                logger.warning('Run time of job "%s" was missed by %s',
-                               job, difference)
-            else:
-                try:
-                    job.add_instance()
-                except MaxInstancesReachedError:
-                    event = JobEvent(EVENT_JOB_MISSED, job, run_time)
-                    self._notify_listeners(event)
-                    logger.warning('Execution of job "%s" skipped: '
-                                   'maximum number of running instances '
-                                   'reached (%d)', job, job.max_instances)
-                    break
-
-                logger.info('Running job "%s" (scheduled at %s)', job,
-                            run_time)
-
-                try:
-                    retval = job.func(*job.args, **job.kwargs)
-                except:
-                    # Notify listeners about the exception
-                    exc, tb = sys.exc_info()[1:]
-                    event = JobEvent(EVENT_JOB_ERROR, job, run_time,
-                                     exception=exc, traceback=tb)
-                    self._notify_listeners(event)
-
-                    logger.exception('Job "%s" raised an exception', job)
-                else:
-                    # Notify listeners about successful execution
-                    event = JobEvent(EVENT_JOB_EXECUTED, job, run_time,
-                                     retval=retval)
-                    self._notify_listeners(event)
-
-                    logger.info('Job "%s" executed successfully', job)
-
-                job.remove_instance()
-
-                # If coalescing is enabled, don't attempt any further runs
-                if job.coalesce:
-                    break
-
-    def _process_jobs(self, now):
-        """
-        Iterates through jobs in every jobstore, starts pending jobs
-        and figures out the next wakeup time.
-        """
-        next_wakeup_time = None
-        self._jobstores_lock.acquire()
-        try:
-            for alias, jobstore in iteritems(self._jobstores):
-                for job in tuple(jobstore.jobs):
-                    run_times = job.get_run_times(now)
-                    if run_times:
-                        self._threadpool.submit(self._run_job, job, run_times)
-
-                        # Increase the job's run count
-                        if job.coalesce:
-                            job.runs += 1
-                        else:
-                            job.runs += len(run_times)
-
-                        # Update the job, but don't keep finished jobs around
-                        if job.compute_next_run_time(now + timedelta(microseconds=1)):
-                            jobstore.update_job(job)
-                        else:
-                            self._remove_job(job, alias, jobstore)
-
-                    if not next_wakeup_time:
-                        next_wakeup_time = job.next_run_time
-                    elif job.next_run_time:
-                        next_wakeup_time = min(next_wakeup_time,
-                                               job.next_run_time)
-            return next_wakeup_time
-        finally:
-            self._jobstores_lock.release()
-
-    def _main_loop(self):
-        """Executes jobs on schedule."""
-
-        logger.info('Scheduler started')
-        self._notify_listeners(SchedulerEvent(EVENT_SCHEDULER_START))
-
-        self._wakeup.clear()
-        while not self._stopped:
-            logger.debug('Looking for jobs to run')
-            now = datetime.now()
-            next_wakeup_time = self._process_jobs(now)
-
-            # Sleep until the next job is scheduled to be run,
-            # a new job is added or the scheduler is stopped
-            if next_wakeup_time is not None:
-                wait_seconds = time_difference(next_wakeup_time, now)
-                logger.debug('Next wakeup is due at %s (in %f seconds)',
-                             next_wakeup_time, wait_seconds)
-                self._wakeup.wait(wait_seconds)
-            else:
-                logger.debug('No jobs; waiting until a job is added')
-                self._wakeup.wait()
-            self._wakeup.clear()
-
-        logger.info('Scheduler has been shut down')
-        self._notify_listeners(SchedulerEvent(EVENT_SCHEDULER_SHUTDOWN))
@@ -0,0 +1,12 @@
+class SchedulerAlreadyRunningError(Exception):
+    """Raised when attempting to start or configure the scheduler when it's already running."""
+
+    def __str__(self):
+        return 'Scheduler is already running'
+
+
+class SchedulerNotRunningError(Exception):
+    """Raised when attempting to shutdown the scheduler when it's not running."""
+
+    def __str__(self):
+        return 'Scheduler is not running'
@@ -0,0 +1,68 @@
+from __future__ import absolute_import
+from functools import wraps
+
+from apscheduler.schedulers.base import BaseScheduler
+from apscheduler.util import maybe_ref
+
+try:
+    import asyncio
+except ImportError:  # pragma: nocover
+    try:
+        import trollius as asyncio
+    except ImportError:
+        raise ImportError('AsyncIOScheduler requires either Python 3.4 or the asyncio package installed')
+
+
+def run_in_event_loop(func):
+    @wraps(func)
+    def wrapper(self, *args, **kwargs):
+        self._eventloop.call_soon_threadsafe(func, self, *args, **kwargs)
+    return wrapper
+
+
+class AsyncIOScheduler(BaseScheduler):
+    """
+    A scheduler that runs on an asyncio (:pep:`3156`) event loop.
+
+    Extra options:
+
+    ============== =============================================================
+    ``event_loop`` AsyncIO event loop to use (defaults to the global event loop)
+    ============== =============================================================
+    """
+
+    _eventloop = None
+    _timeout = None
+
+    def start(self):
+        super(AsyncIOScheduler, self).start()
+        self.wakeup()
+
+    @run_in_event_loop
+    def shutdown(self, wait=True):
+        super(AsyncIOScheduler, self).shutdown(wait)
+        self._stop_timer()
+
+    def _configure(self, config):
+        self._eventloop = maybe_ref(config.pop('event_loop', None)) or asyncio.get_event_loop()
+        super(AsyncIOScheduler, self)._configure(config)
+
+    def _start_timer(self, wait_seconds):
+        self._stop_timer()
+        if wait_seconds is not None:
+            self._timeout = self._eventloop.call_later(wait_seconds, self.wakeup)
+
+    def _stop_timer(self):
+        if self._timeout:
+            self._timeout.cancel()
+            del self._timeout
+
+    @run_in_event_loop
+    def wakeup(self):
+        self._stop_timer()
+        wait_seconds = self._process_jobs()
+        self._start_timer(wait_seconds)
+
+    def _create_default_executor(self):
+        from apscheduler.executors.asyncio import AsyncIOExecutor
+        return AsyncIOExecutor()
@@ -0,0 +1,39 @@
+from __future__ import absolute_import
+from threading import Thread, Event
+
+from apscheduler.schedulers.base import BaseScheduler
+from apscheduler.schedulers.blocking import BlockingScheduler
+from apscheduler.util import asbool
+
+
+class BackgroundScheduler(BlockingScheduler):
+    """
+    A scheduler that runs in the background using a separate thread
+    (:meth:`~apscheduler.schedulers.base.BaseScheduler.start` will return immediately).
+
+    Extra options:
+
+    ========== ============================================================================================
+    ``daemon`` Set the ``daemon`` option in the background thread (defaults to ``True``,
+               see `the documentation <https://docs.python.org/3.4/library/threading.html#thread-objects>`_
+               for further details)
+    ========== ============================================================================================
+    """
+
+    _thread = None
+
+    def _configure(self, config):
+        self._daemon = asbool(config.pop('daemon', True))
+        super(BackgroundScheduler, self)._configure(config)
+
+    def start(self):
+        BaseScheduler.start(self)
+        self._event = Event()
+        self._thread = Thread(target=self._main_loop, name='APScheduler')
+        self._thread.daemon = self._daemon
+        self._thread.start()
+
+    def shutdown(self, wait=True):
+        super(BackgroundScheduler, self).shutdown(wait)
+        self._thread.join()
+        del self._thread
@@ -0,0 +1,845 @@
+from __future__ import print_function
+from abc import ABCMeta, abstractmethod
+from collections import MutableMapping
+from threading import RLock
+from datetime import datetime
+from logging import getLogger
+import sys
+
+from pkg_resources import iter_entry_points
+from tzlocal import get_localzone
+import six
+
+from apscheduler.schedulers import SchedulerAlreadyRunningError, SchedulerNotRunningError
+from apscheduler.executors.base import MaxInstancesReachedError, BaseExecutor
+from apscheduler.executors.pool import ThreadPoolExecutor
+from apscheduler.jobstores.base import ConflictingIdError, JobLookupError, BaseJobStore
+from apscheduler.jobstores.memory import MemoryJobStore
+from apscheduler.job import Job
+from apscheduler.triggers.base import BaseTrigger
+from apscheduler.util import asbool, asint, astimezone, maybe_ref, timedelta_seconds, undefined
+from apscheduler.events import (
+    SchedulerEvent, JobEvent, EVENT_SCHEDULER_START, EVENT_SCHEDULER_SHUTDOWN, EVENT_JOBSTORE_ADDED,
+    EVENT_JOBSTORE_REMOVED, EVENT_ALL, EVENT_JOB_MODIFIED, EVENT_JOB_REMOVED, EVENT_JOB_ADDED, EVENT_EXECUTOR_ADDED,
+    EVENT_EXECUTOR_REMOVED, EVENT_ALL_JOBS_REMOVED)
+
+
+class BaseScheduler(six.with_metaclass(ABCMeta)):
+    """
+    Abstract base class for all schedulers. Takes the following keyword arguments:
+
+    :param str|logging.Logger logger: logger to use for the scheduler's logging (defaults to apscheduler.scheduler)
+    :param str|datetime.tzinfo timezone: the default time zone (defaults to the local timezone)
+    :param dict job_defaults: default values for newly added jobs
+    :param dict jobstores: a dictionary of job store alias -> job store instance or configuration dict
+    :param dict executors: a dictionary of executor alias -> executor instance or configuration dict
+
+    .. seealso:: :ref:`scheduler-config`
+    """
+
+    _trigger_plugins = dict((ep.name, ep) for ep in iter_entry_points('apscheduler.triggers'))
+    _trigger_classes = {}
+    _executor_plugins = dict((ep.name, ep) for ep in iter_entry_points('apscheduler.executors'))
+    _executor_classes = {}
+    _jobstore_plugins = dict((ep.name, ep) for ep in iter_entry_points('apscheduler.jobstores'))
+    _jobstore_classes = {}
+    _stopped = True
+
+    #
+    # Public API
+    #
+
+    def __init__(self, gconfig={}, **options):
+        super(BaseScheduler, self).__init__()
+        self._executors = {}
+        self._executors_lock = self._create_lock()
+        self._jobstores = {}
+        self._jobstores_lock = self._create_lock()
+        self._listeners = []
+        self._listeners_lock = self._create_lock()
+        self._pending_jobs = []
+        self.configure(gconfig, **options)
+
+    def configure(self, gconfig={}, prefix='apscheduler.', **options):
+        """
+        Reconfigures the scheduler with the given options. Can only be done when the scheduler isn't running.
+
+        :param dict gconfig: a "global" configuration dictionary whose values can be overridden by keyword arguments to
+                             this method
+        :param str|unicode prefix: pick only those keys from ``gconfig`` that are prefixed with this string
+                                   (pass an empty string or ``None`` to use all keys)
+        :raises SchedulerAlreadyRunningError: if the scheduler is already running
+        """
+
+        if self.running:
+            raise SchedulerAlreadyRunningError
+
+        # If a non-empty prefix was given, strip it from the keys in the global configuration dict
+        if prefix:
+            prefixlen = len(prefix)
+            gconfig = dict((key[prefixlen:], value) for key, value in six.iteritems(gconfig) if key.startswith(prefix))
+
+        # Create a structure from the dotted options (e.g. "a.b.c = d" -> {'a': {'b': {'c': 'd'}}})
+        config = {}
+        for key, value in six.iteritems(gconfig):
+            parts = key.split('.')
+            parent = config
+            key = parts.pop(0)
+            while parts:
+                parent = parent.setdefault(key, {})
+                key = parts.pop(0)
+            parent[key] = value
+
+        # Override any options with explicit keyword arguments
+        config.update(options)
+        self._configure(config)
+
+    @abstractmethod
+    def start(self):
+        """
+        Starts the scheduler. The details of this process depend on the implementation.
+
+        :raises SchedulerAlreadyRunningError: if the scheduler is already running
+        """
+
+        if self.running:
+            raise SchedulerAlreadyRunningError
+
+        with self._executors_lock:
+            # Create a default executor if nothing else is configured
+            if 'default' not in self._executors:
+                self.add_executor(self._create_default_executor(), 'default')
+
+            # Start all the executors
+            for alias, executor in six.iteritems(self._executors):
+                executor.start(self, alias)
+
+        with self._jobstores_lock:
+            # Create a default job store if nothing else is configured
+            if 'default' not in self._jobstores:
+                self.add_jobstore(self._create_default_jobstore(), 'default')
+
+            # Start all the job stores
+            for alias, store in six.iteritems(self._jobstores):
+                store.start(self, alias)
+
+            # Schedule all pending jobs
+            for job, jobstore_alias, replace_existing in self._pending_jobs:
+                self._real_add_job(job, jobstore_alias, replace_existing, False)
+            del self._pending_jobs[:]
+
+        self._stopped = False
+        self._logger.info('Scheduler started')
+
+        # Notify listeners that the scheduler has been started
+        self._dispatch_event(SchedulerEvent(EVENT_SCHEDULER_START))
+
+    @abstractmethod
+    def shutdown(self, wait=True):
+        """
+        Shuts down the scheduler. Does not interrupt any currently running jobs.
+
+        :param bool wait: ``True`` to wait until all currently executing jobs have finished
+        :raises SchedulerNotRunningError: if the scheduler has not been started yet
+        """
+
+        if not self.running:
+            raise SchedulerNotRunningError
+
+        self._stopped = True
+
+        # Shut down all executors
+        for executor in six.itervalues(self._executors):
+            executor.shutdown(wait)
+
+        # Shut down all job stores
+        for jobstore in six.itervalues(self._jobstores):
+            jobstore.shutdown()
+
+        self._logger.info('Scheduler has been shut down')
+        self._dispatch_event(SchedulerEvent(EVENT_SCHEDULER_SHUTDOWN))
+
+    @property
+    def running(self):
+        return not self._stopped
+
+    def add_executor(self, executor, alias='default', **executor_opts):
+        """
+        Adds an executor to this scheduler. Any extra keyword arguments will be passed to the executor plugin's
+        constructor, assuming that the first argument is the name of an executor plugin.
+
+        :param str|unicode|apscheduler.executors.base.BaseExecutor executor: either an executor instance or the name of
+            an executor plugin
+        :param str|unicode alias: alias for the scheduler
+        :raises ValueError: if there is already an executor by the given alias
+        """
+
+        with self._executors_lock:
+            if alias in self._executors:
+                raise ValueError('This scheduler already has an executor by the alias of "%s"' % alias)
+
+            if isinstance(executor, BaseExecutor):
+                self._executors[alias] = executor
+            elif isinstance(executor, six.string_types):
+                self._executors[alias] = executor = self._create_plugin_instance('executor', executor, executor_opts)
+            else:
+                raise TypeError('Expected an executor instance or a string, got %s instead' %
+                                executor.__class__.__name__)
+
+            # Start the executor right away if the scheduler is running
+            if self.running:
+                executor.start(self)
+
+        self._dispatch_event(SchedulerEvent(EVENT_EXECUTOR_ADDED, alias))
+
+    def remove_executor(self, alias, shutdown=True):
+        """
+        Removes the executor by the given alias from this scheduler.
+
+        :param str|unicode alias: alias of the executor
+        :param bool shutdown: ``True`` to shut down the executor after removing it
+        """
+
+        with self._jobstores_lock:
+            executor = self._lookup_executor(alias)
+            del self._executors[alias]
+
+        if shutdown:
+            executor.shutdown()
+
+        self._dispatch_event(SchedulerEvent(EVENT_EXECUTOR_REMOVED, alias))
+
+    def add_jobstore(self, jobstore, alias='default', **jobstore_opts):
+        """
+        Adds a job store to this scheduler. Any extra keyword arguments will be passed to the job store plugin's
+        constructor, assuming that the first argument is the name of a job store plugin.
+
+        :param str|unicode|apscheduler.jobstores.base.BaseJobStore jobstore: job store to be added
+        :param str|unicode alias: alias for the job store
+        :raises ValueError: if there is already a job store by the given alias
+        """
+
+        with self._jobstores_lock:
+            if alias in self._jobstores:
+                raise ValueError('This scheduler already has a job store by the alias of "%s"' % alias)
+
+            if isinstance(jobstore, BaseJobStore):
+                self._jobstores[alias] = jobstore
+            elif isinstance(jobstore, six.string_types):
+                self._jobstores[alias] = jobstore = self._create_plugin_instance('jobstore', jobstore, jobstore_opts)
+            else:
+                raise TypeError('Expected a job store instance or a string, got %s instead' %
+                                jobstore.__class__.__name__)
+
+            # Start the job store right away if the scheduler is running
+            if self.running:
+                jobstore.start(self, alias)
+
+        # Notify listeners that a new job store has been added
+        self._dispatch_event(SchedulerEvent(EVENT_JOBSTORE_ADDED, alias))
+
+        # Notify the scheduler so it can scan the new job store for jobs
+        if self.running:
+            self.wakeup()
+
+    def remove_jobstore(self, alias, shutdown=True):
+        """
+        Removes the job store by the given alias from this scheduler.
+
+        :param str|unicode alias: alias of the job store
+        :param bool shutdown: ``True`` to shut down the job store after removing it
+        """
+
+        with self._jobstores_lock:
+            jobstore = self._lookup_jobstore(alias)
+            del self._jobstores[alias]
+
+        if shutdown:
+            jobstore.shutdown()
+
+        self._dispatch_event(SchedulerEvent(EVENT_JOBSTORE_REMOVED, alias))
+
+    def add_listener(self, callback, mask=EVENT_ALL):
+        """
+        add_listener(callback, mask=EVENT_ALL)
+
+        Adds a listener for scheduler events. When a matching event occurs, ``callback`` is executed with the event
+        object as its sole argument. If the ``mask`` parameter is not provided, the callback will receive events of all
+        types.
+
+        :param callback: any callable that takes one argument
+        :param int mask: bitmask that indicates which events should be listened to
+
+        .. seealso:: :mod:`apscheduler.events`
+        .. seealso:: :ref:`scheduler-events`
+        """
+
+        with self._listeners_lock:
+            self._listeners.append((callback, mask))
+
+    def remove_listener(self, callback):
+        """Removes a previously added event listener."""
+
+        with self._listeners_lock:
+            for i, (cb, _) in enumerate(self._listeners):
+                if callback == cb:
+                    del self._listeners[i]
+
+    def add_job(self, func, trigger=None, args=None, kwargs=None, id=None, name=None, misfire_grace_time=undefined,
+                coalesce=undefined, max_instances=undefined, next_run_time=undefined, jobstore='default',
+                executor='default', replace_existing=False, **trigger_args):
+        """
+        add_job(func, trigger=None, args=None, kwargs=None, id=None, name=None, misfire_grace_time=undefined, \
+            coalesce=undefined, max_instances=undefined, next_run_time=undefined, jobstore='default', \
+            executor='default', replace_existing=False, **trigger_args)
+
+        Adds the given job to the job list and wakes up the scheduler if it's already running.
+
+        Any option that defaults to ``undefined`` will be replaced with the corresponding default value when the job is
+        scheduled (which happens when the scheduler is started, or immediately if the scheduler is already running).
+
+        The ``func`` argument can be given either as a callable object or a textual reference in the
+        ``package.module:some.object`` format, where the first half (separated by ``:``) is an importable module and the
+        second half is a reference to the callable object, relative to the module.
+
+        The ``trigger`` argument can either be:
+          #. the alias name of the trigger (e.g. ``date``, ``interval`` or ``cron``), in which case any extra keyword
+             arguments to this method are passed on to the trigger's constructor
+          #. an instance of a trigger class
+
+        :param func: callable (or a textual reference to one) to run at the given time
+        :param str|apscheduler.triggers.base.BaseTrigger trigger: trigger that determines when ``func`` is called
+        :param list|tuple args: list of positional arguments to call func with
+        :param dict kwargs: dict of keyword arguments to call func with
+        :param str|unicode id: explicit identifier for the job (for modifying it later)
+        :param str|unicode name: textual description of the job
+        :param int misfire_grace_time: seconds after the designated run time that the job is still allowed to be run
+        :param bool coalesce: run once instead of many times if the scheduler determines that the job should be run more
+                              than once in succession
+        :param int max_instances: maximum number of concurrently running instances allowed for this job
+        :param datetime next_run_time: when to first run the job, regardless of the trigger (pass ``None`` to add the
+                                       job as paused)
+        :param str|unicode jobstore: alias of the job store to store the job in
+        :param str|unicode executor: alias of the executor to run the job with
+        :param bool replace_existing: ``True`` to replace an existing job with the same ``id`` (but retain the
+                                      number of runs from the existing one)
+        :rtype: Job
+        """
+
+        job_kwargs = {
+            'trigger': self._create_trigger(trigger, trigger_args),
+            'executor': executor,
+            'func': func,
+            'args': tuple(args) if args is not None else (),
+            'kwargs': dict(kwargs) if kwargs is not None else {},
+            'id': id,
+            'name': name,
+            'misfire_grace_time': misfire_grace_time,
+            'coalesce': coalesce,
+            'max_instances': max_instances,
+            'next_run_time': next_run_time
+        }
+        job_kwargs = dict((key, value) for key, value in six.iteritems(job_kwargs) if value is not undefined)
+        job = Job(self, **job_kwargs)
+
+        # Don't really add jobs to job stores before the scheduler is up and running
+        with self._jobstores_lock:
+            if not self.running:
+                self._pending_jobs.append((job, jobstore, replace_existing))
+                self._logger.info('Adding job tentatively -- it will be properly scheduled when the scheduler starts')
+            else:
+                self._real_add_job(job, jobstore, replace_existing, True)
+
+        return job
+
+    def scheduled_job(self, trigger, args=None, kwargs=None, id=None, name=None, misfire_grace_time=undefined,
+                      coalesce=undefined, max_instances=undefined, next_run_time=undefined, jobstore='default',
+                      executor='default', **trigger_args):
+        """
+        scheduled_job(trigger, args=None, kwargs=None, id=None, name=None, misfire_grace_time=undefined, \
+            coalesce=undefined, max_instances=undefined, next_run_time=undefined, jobstore='default', \
+            executor='default',**trigger_args)
+
+        A decorator version of :meth:`add_job`, except that ``replace_existing`` is always ``True``.
+
+        .. important:: The ``id`` argument must be given if scheduling a job in a persistent job store. The scheduler
+           cannot, however, enforce this requirement.
+        """
+
+        def inner(func):
+            self.add_job(func, trigger, args, kwargs, id, name, misfire_grace_time, coalesce, max_instances,
+                         next_run_time, jobstore, executor, True, **trigger_args)
+            return func
+        return inner
+
+    def modify_job(self, job_id, jobstore=None, **changes):
+        """
+        Modifies the properties of a single job. Modifications are passed to this method as extra keyword arguments.
+
+        :param str|unicode job_id: the identifier of the job
+        :param str|unicode jobstore: alias of the job store that contains the job
+        """
+        with self._jobstores_lock:
+            job, jobstore = self._lookup_job(job_id, jobstore)
+            job._modify(**changes)
+            if jobstore:
+                self._lookup_jobstore(jobstore).update_job(job)
+
+        self._dispatch_event(JobEvent(EVENT_JOB_MODIFIED, job_id, jobstore))
+
+        # Wake up the scheduler since the job's next run time may have been changed
+        self.wakeup()
+
+    def reschedule_job(self, job_id, jobstore=None, trigger=None, **trigger_args):
+        """
+        Constructs a new trigger for a job and updates its next run time.
+        Extra keyword arguments are passed directly to the trigger's constructor.
+
+        :param str|unicode job_id: the identifier of the job
+        :param str|unicode jobstore: alias of the job store that contains the job
+        :param trigger: alias of the trigger type or a trigger instance
+        """
+
+        trigger = self._create_trigger(trigger, trigger_args)
+        now = datetime.now(self.timezone)
+        next_run_time = trigger.get_next_fire_time(None, now)
+        self.modify_job(job_id, jobstore, trigger=trigger, next_run_time=next_run_time)
+
+    def pause_job(self, job_id, jobstore=None):
+        """
+        Causes the given job not to be executed until it is explicitly resumed.
+
+        :param str|unicode job_id: the identifier of the job
+        :param str|unicode jobstore: alias of the job store that contains the job
+        """
+
+        self.modify_job(job_id, jobstore, next_run_time=None)
+
+    def resume_job(self, job_id, jobstore=None):
+        """
+        Resumes the schedule of the given job, or removes the job if its schedule is finished.
+
+        :param str|unicode job_id: the identifier of the job
+        :param str|unicode jobstore: alias of the job store that contains the job
+        """
+
+        with self._jobstores_lock:
+            job, jobstore = self._lookup_job(job_id, jobstore)
+            now = datetime.now(self.timezone)
+            next_run_time = job.trigger.get_next_fire_time(None, now)
+            if next_run_time:
+                self.modify_job(job_id, jobstore, next_run_time=next_run_time)
+            else:
+                self.remove_job(job.id, jobstore)
+
+    def get_jobs(self, jobstore=None, pending=None):
+        """
+        Returns a list of pending jobs (if the scheduler hasn't been started yet) and scheduled jobs, either from a
+        specific job store or from all of them.
+
+        :param str|unicode jobstore: alias of the job store
+        :param bool pending: ``False`` to leave out pending jobs (jobs that are waiting for the scheduler start to be
+                             added to their respective job stores), ``True`` to only include pending jobs, anything else
+                             to return both
+        :rtype: list[Job]
+        """
+
+        with self._jobstores_lock:
+            jobs = []
+
+            if pending is not False:
+                for job, alias, replace_existing in self._pending_jobs:
+                    if jobstore is None or alias == jobstore:
+                        jobs.append(job)
+
+            if pending is not True:
+                for alias, store in six.iteritems(self._jobstores):
+                    if jobstore is None or alias == jobstore:
+                        jobs.extend(store.get_all_jobs())
+
+            return jobs
+
+    def get_job(self, job_id, jobstore=None):
+        """
+        Returns the Job that matches the given ``job_id``.
+
+        :param str|unicode job_id: the identifier of the job
+        :param str|unicode jobstore: alias of the job store that most likely contains the job
+        :return: the Job by the given ID, or ``None`` if it wasn't found
+        :rtype: Job
+        """
+
+        with self._jobstores_lock:
+            try:
+                return self._lookup_job(job_id, jobstore)[0]
+            except JobLookupError:
+                return
+
+    def remove_job(self, job_id, jobstore=None):
+        """
+        Removes a job, preventing it from being run any more.
+
+        :param str|unicode job_id: the identifier of the job
+        :param str|unicode jobstore: alias of the job store that contains the job
+        :raises JobLookupError: if the job was not found
+        """
+
+        with self._jobstores_lock:
+            # Check if the job is among the pending jobs
+            for i, (job, jobstore_alias, replace_existing) in enumerate(self._pending_jobs):
+                if job.id == job_id:
+                    del self._pending_jobs[i]
+                    jobstore = jobstore_alias
+                    break
+            else:
+                # Otherwise, try to remove it from each store until it succeeds or we run out of stores to check
+                for alias, store in six.iteritems(self._jobstores):
+                    if jobstore in (None, alias):
+                        try:
+                            store.remove_job(job_id)
+                        except JobLookupError:
+                            continue
+
+                        jobstore = alias
+                        break
+
+        if jobstore is None:
+            raise JobLookupError(job_id)
+
+        # Notify listeners that a job has been removed
+        event = JobEvent(EVENT_JOB_REMOVED, job_id, jobstore)
+        self._dispatch_event(event)
+
+        self._logger.info('Removed job %s', job_id)
+
+    def remove_all_jobs(self, jobstore=None):
+        """
+        Removes all jobs from the specified job store, or all job stores if none is given.
+
+        :param str|unicode jobstore: alias of the job store
+        """
+
+        with self._jobstores_lock:
+            if jobstore:
+                self._pending_jobs = [pending for pending in self._pending_jobs if pending[1] != jobstore]
+            else:
+                self._pending_jobs = []
+
+            for alias, store in six.iteritems(self._jobstores):
+                if jobstore in (None, alias):
+                    store.remove_all_jobs()
+
+        self._dispatch_event(SchedulerEvent(EVENT_ALL_JOBS_REMOVED, jobstore))
+
+    def print_jobs(self, jobstore=None, out=None):
+        """
+        print_jobs(jobstore=None, out=sys.stdout)
+
+        Prints out a textual listing of all jobs currently scheduled on either all job stores or just a specific one.
+
+        :param str|unicode jobstore: alias of the job store, ``None`` to list jobs from all stores
+        :param file out: a file-like object to print to (defaults to **sys.stdout** if nothing is given)
+        """
+
+        out = out or sys.stdout
+        with self._jobstores_lock:
+            if self._pending_jobs:
+                print(six.u('Pending jobs:'), file=out)
+                for job, jobstore_alias, replace_existing in self._pending_jobs:
+                    if jobstore in (None, jobstore_alias):
+                        print(six.u('    %s') % job, file=out)
+
+            for alias, store in six.iteritems(self._jobstores):
+                if jobstore in (None, alias):
+                    print(six.u('Jobstore %s:') % alias, file=out)
+                    jobs = store.get_all_jobs()
+                    if jobs:
+                        for job in jobs:
+                            print(six.u('    %s') % job, file=out)
+                    else:
+                        print(six.u('    No scheduled jobs'), file=out)
+
+    @abstractmethod
+    def wakeup(self):
+        """
+        Notifies the scheduler that there may be jobs due for execution.
+        Triggers :meth:`_process_jobs` to be run in an implementation specific manner.
+        """
+
+    #
+    # Private API
+    #
+
+    def _configure(self, config):
+        # Set general options
+        self._logger = maybe_ref(config.pop('logger', None)) or getLogger('apscheduler.scheduler')
+        self.timezone = astimezone(config.pop('timezone', None)) or get_localzone()
+
+        # Set the job defaults
+        job_defaults = config.get('job_defaults', {})
+        self._job_defaults = {
+            'misfire_grace_time': asint(job_defaults.get('misfire_grace_time', 1)),
+            'coalesce': asbool(job_defaults.get('coalesce', True)),
+            'max_instances': asint(job_defaults.get('max_instances', 1))
+        }
+
+        # Configure executors
+        self._executors.clear()
+        for alias, value in six.iteritems(config.get('executors', {})):
+            if isinstance(value, BaseExecutor):
+                self.add_executor(value, alias)
+            elif isinstance(value, MutableMapping):
+                executor_class = value.pop('class', None)
+                plugin = value.pop('type', None)
+                if plugin:
+                    executor = self._create_plugin_instance('executor', plugin, value)
+                elif executor_class:
+                    cls = maybe_ref(executor_class)
+                    executor = cls(**value)
+                else:
+                    raise ValueError('Cannot create executor "%s" -- either "type" or "class" must be defined' % alias)
+
+                self.add_executor(executor, alias)
+            else:
+                raise TypeError("Expected executor instance or dict for executors['%s'], got %s instead" % (
+                    alias, value.__class__.__name__))
+
+        # Configure job stores
+        self._jobstores.clear()
+        for alias, value in six.iteritems(config.get('jobstores', {})):
+            if isinstance(value, BaseJobStore):
+                self.add_jobstore(value, alias)
+            elif isinstance(value, MutableMapping):
+                jobstore_class = value.pop('class', None)
+                plugin = value.pop('type', None)
+                if plugin:
+                    jobstore = self._create_plugin_instance('jobstore', plugin, value)
+                elif jobstore_class:
+                    cls = maybe_ref(jobstore_class)
+                    jobstore = cls(**value)
+                else:
+                    raise ValueError('Cannot create job store "%s" -- either "type" or "class" must be defined' % alias)
+
+                self.add_jobstore(jobstore, alias)
+            else:
+                raise TypeError("Expected job store instance or dict for jobstores['%s'], got %s instead" % (
+                    alias, value.__class__.__name__))
+
+    def _create_default_executor(self):
+        """Creates a default executor store, specific to the particular scheduler type."""
+
+        return ThreadPoolExecutor()
+
+    def _create_default_jobstore(self):
+        """Creates a default job store, specific to the particular scheduler type."""
+
+        return MemoryJobStore()
+
+    def _lookup_executor(self, alias):
+        """
+        Returns the executor instance by the given name from the list of executors that were added to this scheduler.
+
+        :type alias: str
+        :raises KeyError: if no executor by the given alias is not found
+        """
+
+        try:
+            return self._executors[alias]
+        except KeyError:
+            raise KeyError('No such executor: %s' % alias)
+
+    def _lookup_jobstore(self, alias):
+        """
+        Returns the job store instance by the given name from the list of job stores that were added to this scheduler.
+
+        :type alias: str
+        :raises KeyError: if no job store by the given alias is not found
+        """
+
+        try:
+            return self._jobstores[alias]
+        except KeyError:
+            raise KeyError('No such job store: %s' % alias)
+
+    def _lookup_job(self, job_id, jobstore_alias):
+        """
+        Finds a job by its ID.
+
+        :type job_id: str
+        :param str jobstore_alias: alias of a job store to look in
+        :return tuple[Job, str]: a tuple of job, jobstore alias (jobstore alias is None in case of a pending job)
+        :raises JobLookupError: if no job by the given ID is found.
+        """
+
+        # Check if the job is among the pending jobs
+        for job, alias, replace_existing in self._pending_jobs:
+            if job.id == job_id:
+                return job, None
+
+        # Look in all job stores
+        for alias, store in six.iteritems(self._jobstores):
+            if jobstore_alias in (None, alias):
+                job = store.lookup_job(job_id)
+                if job is not None:
+                    return job, alias
+
+        raise JobLookupError(job_id)
+
+    def _dispatch_event(self, event):
+        """
+        Dispatches the given event to interested listeners.
+
+        :param SchedulerEvent event: the event to send
+        """
+
+        with self._listeners_lock:
+            listeners = tuple(self._listeners)
+
+        for cb, mask in listeners:
+            if event.code & mask:
+                try:
+                    cb(event)
+                except:
+                    self._logger.exception('Error notifying listener')
+
+    def _real_add_job(self, job, jobstore_alias, replace_existing, wakeup):
+        """
+        :param Job job: the job to add
+        :param bool replace_existing: ``True`` to use update_job() in case the job already exists in the store
+        :param bool wakeup: ``True`` to wake up the scheduler after adding the job
+        """
+
+        # Fill in undefined values with defaults
+        replacements = {}
+        for key, value in six.iteritems(self._job_defaults):
+            if not hasattr(job, key):
+                replacements[key] = value
+
+        # Calculate the next run time if there is none defined
+        if not hasattr(job, 'next_run_time'):
+            now = datetime.now(self.timezone)
+            replacements['next_run_time'] = job.trigger.get_next_fire_time(None, now)
+
+        # Apply any replacements
+        job._modify(**replacements)
+
+        # Add the job to the given job store
+        store = self._lookup_jobstore(jobstore_alias)
+        try:
+            store.add_job(job)
+        except ConflictingIdError:
+            if replace_existing:
+                store.update_job(job)
+            else:
+                raise
+
+        # Mark the job as no longer pending
+        job._jobstore_alias = jobstore_alias
+
+        # Notify listeners that a new job has been added
+        event = JobEvent(EVENT_JOB_ADDED, job.id, jobstore_alias)
+        self._dispatch_event(event)
+
+        self._logger.info('Added job "%s" to job store "%s"', job.name, jobstore_alias)
+
+        # Notify the scheduler about the new job
+        if wakeup:
+            self.wakeup()
+
+    def _create_plugin_instance(self, type_, alias, constructor_kwargs):
+        """Creates an instance of the given plugin type, loading the plugin first if necessary."""
+
+        plugin_container, class_container, base_class = {
+            'trigger': (self._trigger_plugins, self._trigger_classes, BaseTrigger),
+            'jobstore': (self._jobstore_plugins, self._jobstore_classes, BaseJobStore),
+            'executor': (self._executor_plugins, self._executor_classes, BaseExecutor)
+        }[type_]
+
+        try:
+            plugin_cls = class_container[alias]
+        except KeyError:
+            if alias in plugin_container:
+                plugin_cls = class_container[alias] = plugin_container[alias].load()
+                if not issubclass(plugin_cls, base_class):
+                    raise TypeError('The {0} entry point does not point to a {0} class'.format(type_))
+            else:
+                raise LookupError('No {0} by the name "{1}" was found'.format(type_, alias))
+
+        return plugin_cls(**constructor_kwargs)
+
+    def _create_trigger(self, trigger, trigger_args):
+        if isinstance(trigger, BaseTrigger):
+            return trigger
+        elif trigger is None:
+            trigger = 'date'
+        elif not isinstance(trigger, six.string_types):
+            raise TypeError('Expected a trigger instance or string, got %s instead' % trigger.__class__.__name__)
+
+        # Use the scheduler's time zone if nothing else is specified
+        trigger_args.setdefault('timezone', self.timezone)
+
+        # Instantiate the trigger class
+        return self._create_plugin_instance('trigger', trigger, trigger_args)
+
+    def _create_lock(self):
+        """Creates a reentrant lock object."""
+
+        return RLock()
+
+    def _process_jobs(self):
+        """
+        Iterates through jobs in every jobstore, starts jobs that are due and figures out how long to wait for the next
+        round.
+        """
+
+        self._logger.debug('Looking for jobs to run')
+        now = datetime.now(self.timezone)
+        next_wakeup_time = None
+
+        with self._jobstores_lock:
+            for jobstore_alias, jobstore in six.iteritems(self._jobstores):
+                for job in jobstore.get_due_jobs(now):
+                    # Look up the job's executor
+                    try:
+                        executor = self._lookup_executor(job.executor)
+                    except:
+                        self._logger.error(
+                            'Executor lookup ("%s") failed for job "%s" -- removing it from the job store',
+                            job.executor, job)
+                        self.remove_job(job.id, jobstore_alias)
+                        continue
+
+                    run_times = job._get_run_times(now)
+                    run_times = run_times[-1:] if run_times and job.coalesce else run_times
+                    if run_times:
+                        try:
+                            executor.submit_job(job, run_times)
+                        except MaxInstancesReachedError:
+                            self._logger.warning(
+                                'Execution of job "%s" skipped: maximum number of running instances reached (%d)',
+                                job, job.max_instances)
+                        except:
+                            self._logger.exception('Error submitting job "%s" to executor "%s"', job, job.executor)
+
+                        # Update the job if it has a next execution time. Otherwise remove it from the job store.
+                        job_next_run = job.trigger.get_next_fire_time(run_times[-1], now)
+                        if job_next_run:
+                            job._modify(next_run_time=job_next_run)
+                            jobstore.update_job(job)
+                        else:
+                            self.remove_job(job.id, jobstore_alias)
+
+                # Set a new next wakeup time if there isn't one yet or the jobstore has an even earlier one
+                jobstore_next_run_time = jobstore.get_next_run_time()
+                if jobstore_next_run_time and (next_wakeup_time is None or jobstore_next_run_time < next_wakeup_time):
+                    next_wakeup_time = jobstore_next_run_time
+
+        # Determine the delay until this method should be called again
+        if next_wakeup_time is not None:
+            wait_seconds = max(timedelta_seconds(next_wakeup_time - now), 0)
+            self._logger.debug('Next wakeup is due at %s (in %f seconds)', next_wakeup_time, wait_seconds)
+        else:
+            wait_seconds = None
+            self._logger.debug('No jobs; waiting until a job is added')
+
+        return wait_seconds
@@ -0,0 +1,32 @@
+from __future__ import absolute_import
+from threading import Event
+
+from apscheduler.schedulers.base import BaseScheduler
+
+
+class BlockingScheduler(BaseScheduler):
+    """
+    A scheduler that runs in the foreground (:meth:`~apscheduler.schedulers.base.BaseScheduler.start` will block).
+    """
+
+    MAX_WAIT_TIME = 4294967  # Maximum value accepted by Event.wait() on Windows
+
+    _event = None
+
+    def start(self):
+        super(BlockingScheduler, self).start()
+        self._event = Event()
+        self._main_loop()
+
+    def shutdown(self, wait=True):
+        super(BlockingScheduler, self).shutdown(wait)
+        self._event.set()
+
+    def _main_loop(self):
+        while self.running:
+            wait_seconds = self._process_jobs()
+            self._event.wait(wait_seconds if wait_seconds is not None else self.MAX_WAIT_TIME)
+            self._event.clear()
+
+    def wakeup(self):
+        self._event.set()
@@ -0,0 +1,35 @@
+from __future__ import absolute_import
+
+from apscheduler.schedulers.blocking import BlockingScheduler
+from apscheduler.schedulers.base import BaseScheduler
+
+try:
+    from gevent.event import Event
+    from gevent.lock import RLock
+    import gevent
+except ImportError:  # pragma: nocover
+    raise ImportError('GeventScheduler requires gevent installed')
+
+
+class GeventScheduler(BlockingScheduler):
+    """A scheduler that runs as a Gevent greenlet."""
+
+    _greenlet = None
+
+    def start(self):
+        BaseScheduler.start(self)
+        self._event = Event()
+        self._greenlet = gevent.spawn(self._main_loop)
+        return self._greenlet
+
+    def shutdown(self, wait=True):
+        super(GeventScheduler, self).shutdown(wait)
+        self._greenlet.join()
+        del self._greenlet
+
+    def _create_lock(self):
+        return RLock()
+
+    def _create_default_executor(self):
+        from apscheduler.executors.gevent import GeventExecutor
+        return GeventExecutor()
@@ -0,0 +1,46 @@
+from __future__ import absolute_import
+
+from apscheduler.schedulers.base import BaseScheduler
+
+try:
+    from PyQt5.QtCore import QObject, QTimer
+except ImportError:  # pragma: nocover
+    try:
+        from PyQt4.QtCore import QObject, QTimer
+    except ImportError:
+        try:
+            from PySide.QtCore import QObject, QTimer  # flake8: noqa
+        except ImportError:
+            raise ImportError('QtScheduler requires either PyQt5, PyQt4 or PySide installed')
+
+
+class QtScheduler(BaseScheduler):
+    """A scheduler that runs in a Qt event loop."""
+
+    _timer = None
+
+    def start(self):
+        super(QtScheduler, self).start()
+        self.wakeup()
+
+    def shutdown(self, wait=True):
+        super(QtScheduler, self).shutdown(wait)
+        self._stop_timer()
+
+    def _start_timer(self, wait_seconds):
+        self._stop_timer()
+        if wait_seconds is not None:
+            self._timer = QTimer.singleShot(wait_seconds * 1000, self._process_jobs)
+
+    def _stop_timer(self):
+        if self._timer:
+            if self._timer.isActive():
+                self._timer.stop()
+            del self._timer
+
+    def wakeup(self):
+        self._start_timer(0)
+
+    def _process_jobs(self):
+        wait_seconds = super(QtScheduler, self)._process_jobs()
+        self._start_timer(wait_seconds)
@@ -0,0 +1,60 @@
+from __future__ import absolute_import
+from datetime import timedelta
+from functools import wraps
+
+from apscheduler.schedulers.base import BaseScheduler
+from apscheduler.util import maybe_ref
+
+try:
+    from tornado.ioloop import IOLoop
+except ImportError:  # pragma: nocover
+    raise ImportError('TornadoScheduler requires tornado installed')
+
+
+def run_in_ioloop(func):
+    @wraps(func)
+    def wrapper(self, *args, **kwargs):
+        self._ioloop.add_callback(func, self, *args, **kwargs)
+    return wrapper
+
+
+class TornadoScheduler(BaseScheduler):
+    """
+    A scheduler that runs on a Tornado IOLoop.
+
+    =========== ===============================================================
+    ``io_loop`` Tornado IOLoop instance to use (defaults to the global IO loop)
+    =========== ===============================================================
+    """
+
+    _ioloop = None
+    _timeout = None
+
+    def start(self):
+        super(TornadoScheduler, self).start()
+        self.wakeup()
+
+    @run_in_ioloop
+    def shutdown(self, wait=True):
+        super(TornadoScheduler, self).shutdown(wait)
+        self._stop_timer()
+
+    def _configure(self, config):
+        self._ioloop = maybe_ref(config.pop('io_loop', None)) or IOLoop.current()
+        super(TornadoScheduler, self)._configure(config)
+
+    def _start_timer(self, wait_seconds):
+        self._stop_timer()
+        if wait_seconds is not None:
+            self._timeout = self._ioloop.add_timeout(timedelta(seconds=wait_seconds), self.wakeup)
+
+    def _stop_timer(self):
+        if self._timeout:
+            self._ioloop.remove_timeout(self._timeout)
+            del self._timeout
+
+    @run_in_ioloop
+    def wakeup(self):
+        self._stop_timer()
+        wait_seconds = self._process_jobs()
+        self._start_timer(wait_seconds)
@@ -0,0 +1,65 @@
+from __future__ import absolute_import
+from functools import wraps
+
+from apscheduler.schedulers.base import BaseScheduler
+from apscheduler.util import maybe_ref
+
+try:
+    from twisted.internet import reactor as default_reactor
+except ImportError:  # pragma: nocover
+    raise ImportError('TwistedScheduler requires Twisted installed')
+
+
+def run_in_reactor(func):
+    @wraps(func)
+    def wrapper(self, *args, **kwargs):
+        self._reactor.callFromThread(func, self, *args, **kwargs)
+    return wrapper
+
+
+class TwistedScheduler(BaseScheduler):
+    """
+    A scheduler that runs on a Twisted reactor.
+
+    Extra options:
+
+    =========== ========================================================
+    ``reactor`` Reactor instance to use (defaults to the global reactor)
+    =========== ========================================================
+    """
+
+    _reactor = None
+    _delayedcall = None
+
+    def _configure(self, config):
+        self._reactor = maybe_ref(config.pop('reactor', default_reactor))
+        super(TwistedScheduler, self)._configure(config)
+
+    def start(self):
+        super(TwistedScheduler, self).start()
+        self.wakeup()
+
+    @run_in_reactor
+    def shutdown(self, wait=True):
+        super(TwistedScheduler, self).shutdown(wait)
+        self._stop_timer()
+
+    def _start_timer(self, wait_seconds):
+        self._stop_timer()
+        if wait_seconds is not None:
+            self._delayedcall = self._reactor.callLater(wait_seconds, self.wakeup)
+
+    def _stop_timer(self):
+        if self._delayedcall and self._delayedcall.active():
+            self._delayedcall.cancel()
+            del self._delayedcall
+
+    @run_in_reactor
+    def wakeup(self):
+        self._stop_timer()
+        wait_seconds = self._process_jobs()
+        self._start_timer(wait_seconds)
+
+    def _create_default_executor(self):
+        from apscheduler.executors.twisted import TwistedExecutor
+        return TwistedExecutor()
@@ -1,133 +0,0 @@
-"""
-Generic thread pool class. Modeled after Java's ThreadPoolExecutor.
-Please note that this ThreadPool does *not* fully implement the PEP 3148
-ThreadPool!
-"""
-
-from threading import Thread, Lock, currentThread
-from weakref import ref
-import logging
-import atexit
-
-try:
-    from queue import Queue, Empty
-except ImportError:
-    from Queue import Queue, Empty
-
-logger = logging.getLogger(__name__)
-_threadpools = set()
-
-
-# Worker threads are daemonic in order to let the interpreter exit without
-# an explicit shutdown of the thread pool. The following trick is necessary
-# to allow worker threads to finish cleanly.
-def _shutdown_all():
-    for pool_ref in tuple(_threadpools):
-        pool = pool_ref()
-        if pool:
-            pool.shutdown()
-
-atexit.register(_shutdown_all)
-
-
-class ThreadPool(object):
-    def __init__(self, core_threads=0, max_threads=20, keepalive=1):
-        """
-        :param core_threads: maximum number of persistent threads in the pool
-        :param max_threads: maximum number of total threads in the pool
-        :param thread_class: callable that creates a Thread object
-        :param keepalive: seconds to keep non-core worker threads waiting
-            for new tasks
-        """
-        self.core_threads = core_threads
-        self.max_threads = max(max_threads, core_threads, 1)
-        self.keepalive = keepalive
-        self._queue = Queue()
-        self._threads_lock = Lock()
-        self._threads = set()
-        self._shutdown = False
-
-        _threadpools.add(ref(self))
-        logger.info('Started thread pool with %d core threads and %s maximum '
-                    'threads', core_threads, max_threads or 'unlimited')
-
-    def _adjust_threadcount(self):
-        self._threads_lock.acquire()
-        try:
-            if self.num_threads < self.max_threads:
-                self._add_thread(self.num_threads < self.core_threads)
-        finally:
-            self._threads_lock.release()
-
-    def _add_thread(self, core):
-        t = Thread(target=self._run_jobs, args=(core,))
-        t.setDaemon(True)
-        t.start()
-        self._threads.add(t)
-
-    def _run_jobs(self, core):
-        logger.debug('Started worker thread')
-        block = True
-        timeout = None
-        if not core:
-            block = self.keepalive > 0
-            timeout = self.keepalive
-
-        while True:
-            try:
-                func, args, kwargs = self._queue.get(block, timeout)
-            except Empty:
-                break
-
-            if self._shutdown:
-                break
-
-            try:
-                func(*args, **kwargs)
-            except:
-                logger.exception('Error in worker thread')
-
-        self._threads_lock.acquire()
-        self._threads.remove(currentThread())
-        self._threads_lock.release()
-
-        logger.debug('Exiting worker thread')
-
-    @property
-    def num_threads(self):
-        return len(self._threads)
-
-    def submit(self, func, *args, **kwargs):
-        if self._shutdown:
-            raise RuntimeError('Cannot schedule new tasks after shutdown')
-
-        self._queue.put((func, args, kwargs))
-        self._adjust_threadcount()
-
-    def shutdown(self, wait=True):
-        if self._shutdown:
-            return
-
-        logging.info('Shutting down thread pool')
-        self._shutdown = True
-        _threadpools.remove(ref(self))
-
-        self._threads_lock.acquire()
-        for _ in range(self.num_threads):
-            self._queue.put((None, None, None))
-        self._threads_lock.release()
-
-        if wait:
-            self._threads_lock.acquire()
-            threads = tuple(self._threads)
-            self._threads_lock.release()
-            for thread in threads:
-                thread.join()
-
-    def __repr__(self):
-        if self.max_threads:
-            threadcount = '%d/%d' % (self.num_threads, self.max_threads)
-        else:
-            threadcount = '%d' % self.num_threads
-
-        return '<ThreadPool at %x; threads=%s>' % (id(self), threadcount)
@@ -1,3 +0,0 @@
-from apscheduler.triggers.cron import CronTrigger
-from apscheduler.triggers.interval import IntervalTrigger
-from apscheduler.triggers.simple import SimpleTrigger
@@ -0,0 +1,16 @@
+from abc import ABCMeta, abstractmethod
+
+import six
+
+
+class BaseTrigger(six.with_metaclass(ABCMeta)):
+    """Abstract base class that defines the interface that every trigger must implement."""
+
+    @abstractmethod
+    def get_next_fire_time(self, previous_fire_time, now):
+        """
+        Returns the next datetime to fire on, If no such datetime can be calculated, returns ``None``.
+
+        :param datetime.datetime previous_fire_time: the previous time the trigger was fired
+        :param datetime.datetime now: current datetime
+        """
@@ -1,32 +1,71 @@
-from datetime import date, datetime
+from datetime import datetime, timedelta

-from apscheduler.triggers.cron.fields import *
-from apscheduler.util import datetime_ceil, convert_to_datetime
+from tzlocal import get_localzone
+import six
+
+from apscheduler.triggers.base import BaseTrigger
+from apscheduler.triggers.cron.fields import BaseField, WeekField, DayOfMonthField, DayOfWeekField, DEFAULT_VALUES
+from apscheduler.util import datetime_ceil, convert_to_datetime, datetime_repr, astimezone


-class CronTrigger(object):
-    FIELD_NAMES = ('year', 'month', 'day', 'week', 'day_of_week', 'hour',
-                   'minute', 'second')
-    FIELDS_MAP = {'year': BaseField,
-                  'month': BaseField,
-                  'week': WeekField,
-                  'day': DayOfMonthField,
-                  'day_of_week': DayOfWeekField,
-                  'hour': BaseField,
-                  'minute': BaseField,
-                  'second': BaseField}
+class CronTrigger(BaseTrigger):
+    """
+    Triggers when current time matches all specified time constraints, similarly to how the UNIX cron scheduler works.

-    def __init__(self, **values):
-        self.start_date = values.pop('start_date', None)
-        if self.start_date:
-            self.start_date = convert_to_datetime(self.start_date)
+    :param int|str year: 4-digit year
+    :param int|str month: month (1-12)
+    :param int|str day: day of the (1-31)
+    :param int|str week: ISO week (1-53)
+    :param int|str day_of_week: number or name of weekday (0-6 or mon,tue,wed,thu,fri,sat,sun)
+    :param int|str hour: hour (0-23)
+    :param int|str minute: minute (0-59)
+    :param int|str second: second (0-59)
+    :param datetime|str start_date: earliest possible date/time to trigger on (inclusive)
+    :param datetime|str end_date: latest possible date/time to trigger on (inclusive)
+    :param datetime.tzinfo|str timezone: time zone to use for the date/time calculations
+                                         (defaults to scheduler timezone)

+    .. note:: The first weekday is always **monday**.
+    """
+
+    FIELD_NAMES = ('year', 'month', 'day', 'week', 'day_of_week', 'hour', 'minute', 'second')
+    FIELDS_MAP = {
+        'year': BaseField,
+        'month': BaseField,
+        'week': WeekField,
+        'day': DayOfMonthField,
+        'day_of_week': DayOfWeekField,
+        'hour': BaseField,
+        'minute': BaseField,
+        'second': BaseField
+    }
+
+    __slots__ = 'timezone', 'start_date', 'end_date', 'fields'
+
+    def __init__(self, year=None, month=None, day=None, week=None, day_of_week=None, hour=None, minute=None,
+                 second=None, start_date=None, end_date=None, timezone=None):
+        if timezone:
+            self.timezone = astimezone(timezone)
+        elif start_date and start_date.tzinfo:
+            self.timezone = start_date.tzinfo
+        elif end_date and end_date.tzinfo:
+            self.timezone = end_date.tzinfo
+        else:
+            self.timezone = get_localzone()
+
+        self.start_date = convert_to_datetime(start_date, self.timezone, 'start_date')
+        self.end_date = convert_to_datetime(end_date, self.timezone, 'end_date')
+
+        values = dict((key, value) for (key, value) in six.iteritems(locals())
+                      if key in self.FIELD_NAMES and value is not None)
        self.fields = []
+        assign_defaults = False
        for field_name in self.FIELD_NAMES:
            if field_name in values:
                exprs = values.pop(field_name)
                is_default = False
-            elif not values:
+                assign_defaults = not values
+            elif assign_defaults:
                exprs = DEFAULT_VALUES[field_name]
                is_default = True
            else:
@@ -39,18 +78,16 @@ class CronTrigger(object):

    def _increment_field_value(self, dateval, fieldnum):
        """
-        Increments the designated field and resets all less significant fields
-        to their minimum values.
+        Increments the designated field and resets all less significant fields to their minimum values.

        :type dateval: datetime
        :type fieldnum: int
-        :type amount: int
+        :return: a tuple containing the new date, and the number of the field that was actually incremented
        :rtype: tuple
-        :return: a tuple containing the new date, and the number of the field
-                 that was actually incremented
        """
-        i = 0
+
        values = {}
+        i = 0
        while i < len(self.fields):
            field = self.fields[i]
            if not field.REAL:
@@ -77,7 +114,8 @@ class CronTrigger(object):
                    values[field.name] = value + 1
                    i += 1

-        return datetime(**values), fieldnum
+        difference = datetime(**values) - dateval.replace(tzinfo=None)
+        return self.timezone.normalize(dateval + difference), fieldnum

    def _set_field_value(self, dateval, fieldnum, new_value):
        values = {}
@@ -90,13 +128,17 @@ class CronTrigger(object):
                else:
                    values[field.name] = new_value

-        return datetime(**values)
+        difference = datetime(**values) - dateval.replace(tzinfo=None)
+        return self.timezone.normalize(dateval + difference)
+
+    def get_next_fire_time(self, previous_fire_time, now):
+        if previous_fire_time:
+            start_date = max(now, previous_fire_time + timedelta(microseconds=1))
+        else:
+            start_date = max(now, self.start_date) if self.start_date else now

-    def get_next_fire_time(self, start_date):
-        if self.start_date:
-            start_date = max(start_date, self.start_date)
-        next_date = datetime_ceil(start_date)
        fieldnum = 0
+        next_date = datetime_ceil(start_date).astimezone(self.timezone)
        while 0 <= fieldnum < len(self.fields):
            field = self.fields[fieldnum]
            curr_value = field.get_value(next_date)
@@ -104,32 +146,31 @@ class CronTrigger(object):

            if next_value is None:
                # No valid value was found
-                next_date, fieldnum = self._increment_field_value(next_date,
-                                                                  fieldnum - 1)
+                next_date, fieldnum = self._increment_field_value(next_date, fieldnum - 1)
            elif next_value > curr_value:
                # A valid, but higher than the starting value, was found
                if field.REAL:
-                    next_date = self._set_field_value(next_date, fieldnum,
-                                                      next_value)
+                    next_date = self._set_field_value(next_date, fieldnum, next_value)
                    fieldnum += 1
                else:
-                    next_date, fieldnum = self._increment_field_value(next_date,
-                                                                      fieldnum)
+                    next_date, fieldnum = self._increment_field_value(next_date, fieldnum)
            else:
                # A valid value was found, no changes necessary
                fieldnum += 1

+            # Return if the date has rolled past the end date
+            if self.end_date and next_date > self.end_date:
+                return None
+
        if fieldnum >= 0:
            return next_date

    def __str__(self):
-        options = ["%s='%s'" % (f.name, str(f)) for f in self.fields
-                   if not f.is_default]
+        options = ["%s='%s'" % (f.name, f) for f in self.fields if not f.is_default]
        return 'cron[%s]' % (', '.join(options))

    def __repr__(self):
-        options = ["%s='%s'" % (f.name, str(f)) for f in self.fields
-                   if not f.is_default]
+        options = ["%s='%s'" % (f.name, f) for f in self.fields if not f.is_default]
        if self.start_date:
-            options.append("start_date='%s'" % self.start_date.isoformat(' '))
+            options.append("start_date='%s'" % datetime_repr(self.start_date))
        return '<%s (%s)>' % (self.__class__.__name__, ', '.join(options))
--- a/Show More
+++ b/Show More