{"id":1367,"date":"2010-08-25T03:23:26","date_gmt":"2010-08-24T17:23:26","guid":{"rendered":"http:\/\/www.somethinkodd.com\/oddthinking\/"},"modified":"2013-04-20T03:18:08","modified_gmt":"2013-04-19T17:18:08","slug":"nonblocking-log-handler","status":"publish","type":"page","link":"https:\/\/www.somethinkodd.com\/oddthinking\/nonblocking-log-handler\/","title":{"rendered":"Nonblock Log Hndlr"},"content":{"rendered":"<p>This is the official page for the <a href=\"http:\/\/somethinkodd.com\/nonblockingloghandler\">NonBlockingLogHandler Python Package<\/a>.<\/p>\n<h4>Latest Version<\/h4>\n<p>The latest version of the package is available from <a href=\"http:\/\/pypi.python.org\/pypi\/nonblockingloghandler\/\">PyPI<\/a>.<\/p>\n<p>You can download it from there, but if you have <code>easy_install<\/code> available, you can use:<br \/>\n    <code>easy_install nonblockingloghandler<\/code><\/p>\n<p>If you prefer <code>PIP<\/code>, you can use<br \/>\n    <code>pip install nonblockingloghandler<\/code><\/p>\n<h4>Why would I use it?<\/h4>\n<p>You would only need it if:<\/p>\n<ul>\n<li>you are a <em>Python programmer<\/em>,<\/li>\n<li>you are writing a <em>real-time<\/em> system (i.e. one where a late response is a wrong response), AND<\/li>\n<li>you are using the inbuilt Python <em>logging<\/em> function to produce log messages that take some time to emit &#8211; for example, to send emails.<\/li>\n<\/ul>\n<p>If all three apply to you, you may find that your thread is suspended indefinitely while waiting for the log message to be emitted. In this way, the logging system can cause disruption to your code&#8217;s performance.<\/p>\n<p>This component attempts to overcome such issues.<\/p>\n<h4>What Does It Look Like?<\/h4>\n<p>The package offers a <code>NonblockingLogHandler<\/code> class consistent with the Python logging subsystem.<\/p>\n<p>This handler acts as a proxy for the another log handler that may be slow to execute: e.g. the SMTPHandler, SocketHandler, SysLogHandler &#8211; especially when they are talking to remote servers.<\/p>\n<p>The slow log handler is passed as a parameter to the constructor of <code>NonblockingLogHandler<\/code>. The resulting instance is intended to be a drop-in replacement (see provisos below) for the proxied handler. The class returns quickly, and executes the actually logging in the background, in a separate thread.<\/p>\n<h4>Provisos<\/h4>\n<p>While there has been every effort to make the new instance act as transparently as possible for the normal case, the logging module is highly customisable, and it is possible that some customisations will conflict with the <code>NonblockingLogHandler<\/code>.<\/p>\n<h5><code>__str__()<\/code> methods<\/h5>\n<p>If you pass objects to the logging functions (such as <code>info()<\/code> or <code>error()<\/code>, be aware that their <code>__str__()<\/code> methods should be fast. I\/O bound str() calls are outside the scope of this module.<\/p>\n<p>Execution of <code>str()<\/code> functions on message parameters and string formatting is done immediately, in the calling thread. This is to guarantee both thread-safety of the logged objects and also that the log shows the object&#8217;s values at the time of the call, not the time of the emit.<\/p>\n<p>This means, slow <code>__str__()<\/code> methods will affect the performance of the calling code.<\/p>\n<h5>Subclassed Filter<\/h5>\n<p>Most people do not need to subclass from the <code>Filter<\/code> class. If you do subclass it, be aware that they should be fast. I\/O-bound filters are outside of the scope of this module.<\/p>\n<h5>Subclassed Formatter<\/h5>\n<p>Most people do not need to subclass from the <code>Formatter<\/code> class. If you do subclass it, note that it will not have the user parameters from the logging call passed to it. It will only receive the resulting message string already formatted according to the client&#8217;s wishes. The new subclass can still successfully format dates, threadnames, levels, etc.<\/p>\n<h5>The <code>format()<\/code> and <code>emit()<\/code> Methods<\/h5>\n<p>The <code>NonblockingLogHandler<\/code> class provides <code>format()<\/code>S and <code>emit()<\/code> methods for internal-use only. Clients should not directly call them. Consider them &#8220;private&#8221;.<\/p>\n<h5>Separate Levels and Filter States<\/h5>\n<p>Once the NonblockingLogHandler handler is initialised, any further calls to <code>setLevel()<\/code>, <code>addFilter()<\/code> and <code>removeFilter()<\/code> made on the delegated handler are ignored by the proxy handler. Conversely, the delegated handler is not informed of calls made on the <code>NonblockingLogHandler<\/code> instance.<\/p>\n<p>Best practice is to only update the NonblockingLogHandler handler, and not use the values stored in the delegated handler.<\/p>\n<h5>Example Usage<\/h5>\n<blockquote><p><code>  nonblocking_email_handler = nonblockingloghandler.NonblockingLogHandler(<br \/>\n      logging.SMTPHandler(\"localhost\", \"<a  rel=\"nofollow\" id=\"sto_emailShroud0\" href=\"http:\/\/www.somethinkodd.com\/emailshroud\/emailaddress.php?encryptedAddress=moc%40%40metsys_gniggol.elpmaxe&amp;ver=2.2.1\">logging_system<\/a>\", \"<a  rel=\"nofollow\" id=\"sto_emailShroud1\" href=\"http:\/\/www.somethinkodd.com\/emailshroud\/emailaddress.php?encryptedAddress=moc%40%40nimda.elpmaxe&amp;ver=2.2.1\">admin<\/a>\", \"Log message\")<br \/>\n      )<br \/>\n  db_logger = logging.getLogger(\"database\")<br \/>\n  db_logger.addHandler(nonblocking_email_handler)<\/p>\n<p>  db_logger.critical(\"Database corrupted\") # This operation will return immediately, before email is sent.<\/p>\n<p>  nonblocking_email_handler.close()<br \/>\n<\/code><\/p><\/blockquote>\n<h5>Future Versions<\/h5>\n<p>As of Version 1.1.2, there are the following known deficiencies:<\/p>\n<ul>\n<li>It is only tested on Python 2.7.2, on Windows and Ubuntu. If there is demand, I can widen that.<\/li>\n<li>There is a rare and intermittent shut-down race condition. It only occurs on Windows.\n<ul>\n<li>All that happens is, sometimes when your program shuts down immediately after emitting a log message, the Python Runtime emits a warning: &#8220;Exception in thread NonblockingLogHandler.[&#8230;] (most likely raised during interpreter shutdown)&#8221;<\/li>\n<li>I suspect it is the same root cause as a bug that was fixed in the Python 3.2 codebase, but was not backported to Python 2.7. I do not have any strong evidence of this conjecture.<\/li>\n<\/ul>\n<\/li>\n<li>The automated tests require manual inspection to confirm they have run successfully. They should be further automated.<\/li>\n<li>The tests require editing on Ubuntu &#8211; to set the explicit path to Python. This should be transparent.<\/li>\n<\/ul>\n<h5>Version History<\/h5>\n<p>1.1.2 &#8211; Official release 20-Apr-2013. Identifies, but doesn&#8217;t fix, intermittent shut-down bug. Better handling\/clarification of appropriate close() behaviour, which is believed to have been interfering with use in daemons on Ubuntu. Better adherence to PEP8 and PEP 396 standards.<br \/>\n1.0.1 &#8211; Official release &#8211; 6-Jun-2011.<br \/>\n0.3 &#8211; First version to be released &#8211; works, except performance and stability may fail if called in repeatedly and rapidly, due to thread resource usage.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is the official page for the NonBlockingLogHandler Python Package. Latest Version The latest version of the package is available from PyPI. You can download it from there, but if you have easy_install available, you can use: easy_install nonblockingloghandler If you prefer PIP, you can use pip install nonblockingloghandler Why would I use it? You [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":60,"comment_status":"open","ping_status":"open","template":"","meta":{"_s2mail":"yes","footnotes":""},"class_list":["post-1367","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/pages\/1367","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/comments?post=1367"}],"version-history":[{"count":15,"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/pages\/1367\/revisions"}],"predecessor-version":[{"id":1383,"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/pages\/1367\/revisions\/1383"}],"wp:attachment":[{"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/media?parent=1367"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}