:mod:`asynchat` --- Asynchronous socket command/response handler
================================================================

.. module:: asynchat
   :synopsis: Support for asynchronous command/response protocols.
.. moduleauthor:: Sam Rushing <rushing@nightmare.com>
.. sectionauthor:: Steve Holden <sholden@holdenweb.com>


This module builds on the :mod:`asyncore` infrastructure, simplifying
asynchronous clients and servers and making it easier to handle protocols whose
elements are terminated by arbitrary strings, or are of variable length.
:mod:`asynchat` defines the abstract class :class:`async_chat` that you
subclass, providing implementations of the :meth:`collect_incoming_data` and
:meth:`found_terminator` methods. It uses the same asynchronous loop as
:mod:`asyncore`, and the two types of channel, :class:`asyncore.dispatcher` and
:class:`asynchat.async_chat`, can freely be mixed in the channel map. Typically
an :class:`asyncore.dispatcher` server channel generates new
:class:`asynchat.async_chat` channel objects as it receives incoming connection
requests.


.. class:: async_chat()

   This class is an abstract subclass of :class:`asyncore.dispatcher`. To make
   practical use of the code you must subclass :class:`async_chat`, providing
   meaningful :meth:`collect_incoming_data` and :meth:`found_terminator` methods.
   The :class:`asyncore.dispatcher` methods can be used, although not all make
   sense in a message/response context.

   Like :class:`asyncore.dispatcher`, :class:`async_chat` defines a set of events
   that are generated by an analysis of socket conditions after a :cfunc:`select`
   call. Once the polling loop has been started the :class:`async_chat` object's
   methods are called by the event-processing framework with no action on the part
   of the programmer.

   Unlike :class:`asyncore.dispatcher`, :class:`async_chat` allows you to define a
   first-in-first-out queue (fifo) of *producers*. A producer need have only one
   method, :meth:`more`, which should return data to be transmitted on the channel.
   The producer indicates exhaustion (*i.e.* that it contains no more data) by
   having its :meth:`more` method return the empty string. At this point the
   :class:`async_chat` object removes the producer from the fifo and starts using
   the next producer, if any. When the producer fifo is empty the
   :meth:`handle_write` method does nothing. You use the channel object's
   :meth:`set_terminator` method to describe how to recognize the end of, or an
   important breakpoint in, an incoming transmission from the remote endpoint.

   To build a functioning :class:`async_chat` subclass your  input methods
   :meth:`collect_incoming_data` and :meth:`found_terminator` must handle the data
   that the channel receives asynchronously. The methods are described below.


.. method:: async_chat.close_when_done()

   Pushes a ``None`` on to the producer fifo. When this producer is popped off the
   fifo it causes the channel to be closed.


.. method:: async_chat.collect_incoming_data(data)

   Called with *data* holding an arbitrary amount of received data. The default
   method, which must be overridden, raises a :exc:`NotImplementedError` exception.


.. method:: async_chat.discard_buffers()

   In emergencies this method will discard any data held in the input and/or output
   buffers and the producer fifo.


.. method:: async_chat.found_terminator()

   Called when the incoming data stream  matches the termination condition set by
   :meth:`set_terminator`. The default method, which must be overridden, raises a
   :exc:`NotImplementedError` exception. The buffered input data should be
   available via an instance attribute.


.. method:: async_chat.get_terminator()

   Returns the current terminator for the channel.


.. method:: async_chat.handle_close()

   Called when the channel is closed. The default method silently closes the
   channel's socket.


.. method:: async_chat.handle_read()

   Called when a read event fires on the channel's socket in the asynchronous loop.
   The default method checks for the termination condition established by
   :meth:`set_terminator`, which can be either the appearance of a particular
   string in the input stream or the receipt of a particular number of characters.
   When the terminator is found, :meth:`handle_read` calls the
   :meth:`found_terminator` method after calling :meth:`collect_incoming_data` with
   any data preceding the terminating condition.


.. method:: async_chat.handle_write()

   Called when the application may write data to the channel.   The default method
   calls the :meth:`initiate_send` method, which in turn will call
   :meth:`refill_buffer` to collect data from the producer fifo associated with the
   channel.


.. method:: async_chat.push(data)

   Creates a :class:`simple_producer` object (*see below*) containing the data and
   pushes it on to the channel's ``producer_fifo`` to ensure its transmission. This
   is all you need to do to have the channel write the data out to the network,
   although it is possible to use your own producers in more complex schemes to
   implement encryption and chunking, for example.


.. method:: async_chat.push_with_producer(producer)

   Takes a producer object and adds it to the producer fifo associated with the
   channel. When all currently-pushed producers have been exhausted the channel
   will consume this producer's data by calling its :meth:`more` method and send
   the data to the remote endpoint.


.. method:: async_chat.readable()

   Should return ``True`` for the channel to be included in the set of channels
   tested by the :cfunc:`select` loop for readability.


.. method:: async_chat.refill_buffer()

   Refills the output buffer by calling the :meth:`more` method of the producer at
   the head of the fifo. If it is exhausted then the producer is popped off the
   fifo and the next producer is activated. If the current producer is, or becomes,
   ``None`` then the channel is closed.


.. method:: async_chat.set_terminator(term)

   Sets the terminating condition to be recognised on the channel. ``term`` may be
   any of three types of value, corresponding to three different ways to handle
   incoming protocol data.

   +-----------+---------------------------------------------+
   | term      | Description                                 |
   +===========+=============================================+
   | *string*  | Will call :meth:`found_terminator` when the |
   |           | string is found in the input stream         |
   +-----------+---------------------------------------------+
   | *integer* | Will call :meth:`found_terminator` when the |
   |           | indicated number of characters have been    |
   |           | received                                    |
   +-----------+---------------------------------------------+
   | ``None``  | The channel continues to collect data       |
   |           | forever                                     |
   +-----------+---------------------------------------------+

   Note that any data following the terminator will be available for reading by the
   channel after :meth:`found_terminator` is called.


.. method:: async_chat.writable()

   Should return ``True`` as long as items remain on the producer fifo, or the
   channel is connected and the channel's output buffer is non-empty.


asynchat - Auxiliary Classes and Functions
------------------------------------------


.. class:: simple_producer(data[, buffer_size=512])

   A :class:`simple_producer` takes a chunk of data and an optional buffer size.
   Repeated calls to its :meth:`more` method yield successive chunks of the data no
   larger than *buffer_size*.


.. method:: simple_producer.more()

   Produces the next chunk of information from the producer, or returns the empty
   string.


.. class:: fifo([list=None])

   Each channel maintains a :class:`fifo` holding data which has been pushed by the
   application but not yet popped for writing to the channel. A :class:`fifo` is a
   list used to hold data and/or producers until they are required. If the *list*
   argument is provided then it should contain producers or data items to be
   written to the channel.


.. method:: fifo.is_empty()

   Returns ``True`` iff the fifo is empty.


.. method:: fifo.first()

   Returns the least-recently :meth:`push`\ ed item from the fifo.


.. method:: fifo.push(data)

   Adds the given data (which may be a string or a producer object) to the producer
   fifo.


.. method:: fifo.pop()

   If the fifo is not empty, returns ``True, first()``, deleting the popped item.
   Returns ``False, None`` for an empty fifo.

The :mod:`asynchat` module also defines one utility function, which may be of
use in network and textual analysis operations.


.. function:: find_prefix_at_end(haystack, needle)

   Returns ``True`` if string *haystack* ends with any non-empty prefix of string
   *needle*.


.. _asynchat-example:

asynchat Example
----------------

The following partial example shows how HTTP requests can be read with
:class:`async_chat`. A web server might create an :class:`http_request_handler`
object for each incoming client connection. Notice that initially the channel
terminator is set to match the blank line at the end of the HTTP headers, and a
flag indicates that the headers are being read.

Once the headers have been read, if the request is of type POST (indicating that
further data are present in the input stream) then the ``Content-Length:``
header is used to set a numeric terminator to read the right amount of data from
the channel.

The :meth:`handle_request` method is called once all relevant input has been
marshalled, after setting the channel terminator to ``None`` to ensure that any
extraneous data sent by the web client are ignored. ::

   class http_request_handler(asynchat.async_chat):

       def __init__(self, conn, addr, sessions, log):
           asynchat.async_chat.__init__(self, conn=conn)
           self.addr = addr
           self.sessions = sessions
           self.ibuffer = []
           self.obuffer = ""
           self.set_terminator("\r\n\r\n")
           self.reading_headers = True
           self.handling = False
           self.cgi_data = None
           self.log = log

       def collect_incoming_data(self, data):
           """Buffer the data"""
           self.ibuffer.append(data)

       def found_terminator(self):
           if self.reading_headers:
               self.reading_headers = False
               self.parse_headers("".join(self.ibuffer))
               self.ibuffer = []
               if self.op.upper() == "POST":
                   clen = self.headers.getheader("content-length")
                   self.set_terminator(int(clen))
               else:
                   self.handling = True
                   self.set_terminator(None)
                   self.handle_request()
           elif not self.handling:
               self.set_terminator(None) # browsers sometimes over-send
               self.cgi_data = parse(self.headers, "".join(self.ibuffer))
               self.handling = True
               self.ibuffer = []
               self.handle_request()