DOCSP-37005 - Periodic Executors (#12)

Co-authored-by: Jordan Smith <[email protected]>

Author: mongoKart

source/faq.txt

@@ -517,7 +517,7 @@ MongoDB backends for Django sessions and authentication (bypassing
 Does PyMongo Work with mod_wsgi?
 --------------------------------
 
-Yes. See the configuration guide for :ref:`pymongo-and-mod_wsgi`.
+Yes. See the configuration guide for :ref:`pymongo-mod_wsgi`.
 
 Does PyMongo Work with PythonAnywhere?
 --------------------------------------

source/fundamentals.txt

@@ -5,12 +5,14 @@ Fundamentals
 ============
 
 .. meta::
-   :description: Learn how to use the (+driver-short+} to execute operations on MongoDB.
+   :description: Learn how to use (+driver-short+} to execute operations on MongoDB.
 
 .. toctree::
    :titlesonly:
    :maxdepth: 1
 
+   /fundamentals/periodic-executors
    /fundamentals/type-hints
 
+- :ref:`pymongo-periodic-executors`
 - :ref:`pymongo-type-hints`

source/fundamentals/periodic-executors.txt

@@ -1,8 +1,21 @@
-.. uses periodic-executor.rst
+.. _pymongo-periodic-executors:
 
 Periodic Executors
 ==================
 
+.. contents:: On this page
+   :local:
+   :backlinks: none
+   :depth: 1
+   :class: singlecol
+
+.. facet::
+   :name: genre
+   :values: reference
+
+.. meta::
+   :keywords: sleep, timeout, wake
+
 PyMongo implements a ``~periodic_executor.PeriodicExecutor`` for two
 purposes: as the background thread for ``~monitor.Monitor``, and to
 regularly check if there are ``OP_KILL_CURSORS`` messages that must be sent to the server.
@@ -16,44 +29,42 @@ the cursor before finishing iteration:
 
 .. code-block:: python
 
-    for doc in collection.find():
-        raise Exception()
+   for doc in collection.find():
+       raise Exception()
 
-We try to send an ``OP_KILL_CURSORS`` to the server to tell it to clean up the
-server-side cursor. But we must not take any locks directly from the cursor's
-destructor (see `PYTHON-799`_), so we cannot safely use the PyMongo data
-structures required to send a message. The solution is to add the cursor's id
+The client tries to send an ``OP_KILL_CURSORS`` to the server to tell it to clean up the
+server-side cursor. But the client must not take any locks directly from the cursor's
+destructor (see `PYTHON-799 <https://jira.mongodb.org/browse/PYTHON-799>`__
+), so we cannot safely use the PyMongo data
+structures required to send a message. The solution is to add the cursor's ID
 to an array on the ``~mongo_client.MongoClient`` without taking any locks.
 
 Each client has a ``~periodic_executor.PeriodicExecutor`` devoted to
-checking the array for cursor ids. Any it sees are the result of cursors that
+checking the array for cursor IDs. Any it sees are the result of cursors that
 were freed while the server-side cursor was still open. The executor can safely
 take the locks it needs in order to send the ``OP_KILL_CURSORS`` message.
 
-.. _PYTHON-799: https://jira.mongodb.org/browse/PYTHON-799
-
 Stopping Executors
 ------------------
 
 Just as ``~cursor.Cursor`` must not take any locks from its destructor,
 neither can ``~mongo_client.MongoClient`` and ``~topology.Topology``.
-Thus, although the client calls the ``close`` method on its kill-cursors thread, and
-the topology calls the ``close`` method on all its monitor threads, the the ``close`` method
-method cannot actually call the ``wake`` method on the executor, since the ``wake`` method
+Thus, although the client calls the ``close()`` method on its kill-cursors thread, and
+the topology calls the ``close()`` method on all its monitor threads, the ``close()`` method
+method cannot actually call the ``wake()`` method on the executor, since the ``wake()`` method
 takes a lock.
 
 Instead, executors wake periodically to check if ``self.close`` is set,
-and if so they exit.
+and if so, they exit.
 
 A thread can log spurious errors if it wakes late in the Python interpreter's
 shutdown sequence, so we try to join threads before then. Each periodic
 executor (either a monitor or a kill-cursors thread) adds a weakref to itself
 to a set called ``_EXECUTORS``, in the ``periodic_executor`` module.
 
-An `exit handler`_ runs on shutdown and tells all executors to stop, then
-tries (with a short timeout) to join all executor threads.
-
-.. _exit handler: https://docs.python.org/2/library/atexit.html
+An `exit handler <https://docs.python.org/2/library/atexit.html>`__ runs on shutdown
+and tells all executors to stop, then tries (with a short timeout) to join all
+executor threads.
 
 Monitoring
 ----------
@@ -65,52 +76,42 @@ callback to terminate itself soon after the topology is freed.
 
 Solid lines represent strong references, dashed lines weak ones:
 
-.. generated with graphviz: "dot -Tpng periodic-executor-refs.dot > periodic-executor-refs.png"
-
 .. image:: ../images/periodic-executor-refs.png
    :alt: Periodic executor references
 
-See Stopping Executors above for an explanation of the ``_EXECUTORS`` set.
-
-It is a requirement of the `Server Discovery And Monitoring Spec`_ that a
-sleeping monitor can be awakened early. Aside from infrequent wakeups to do
+Aside from infrequent wakeups to do
 their appointed chores, and occasional interruptions, periodic executors also
 wake periodically to check if they should terminate.
 
-Our first implementation of this idea was the obvious one: use the Python
-standard library's threading.Condition.wait with a timeout. Another thread
-wakes the executor early by signaling the condition variable.
+Initially, {+driver-short+} used the Python
+standard library's ``threading.Condition.wait()`` method with a timeout. Another thread
+woke the executor early by signaling the condition variable.
 
 A topology cannot signal the condition variable to tell the executor to
 terminate, because it would risk a deadlock in the garbage collector: no
 destructor or weakref callback can take a lock to signal the condition variable
-(see `PYTHON-863`_); thus the only way for a dying object to terminate a
-periodic executor is to set its "stopped" flag and let the executor see the
-flag next time it wakes.
+(see `PYTHON-863 <https://jira.mongodb.org/browse/PYTHON-863>`__). The only way for a
+dying object to terminate a periodic executor is to set its "stopped" flag and let the
+executor see the flag next time it wakes.
 
 We erred on the side of prompt cleanup, and set the check interval at 100ms. We
 assumed that checking a flag and going back to sleep 10 times a second was
 cheap on modern machines.
 
-Starting in Python 3.2, the builtin C implementation of lock.acquire takes a
-timeout parameter, so Python 3.2+ Condition variables sleep simply by calling
-lock.acquire; they are implemented as efficiently as expected.
+Starting in Python 3.2, the built-in C implementation of the ``lock.acquire()`` method
+takes a ``timeout`` parameter, so Python 3.2+ condition variables sleep simply by calling
+``lock.acquire()``. They are implemented as efficiently as expected.
 
-But in Python 2, lock.acquire has no timeout. To wait with a timeout, a Python
-2 condition variable sleeps a millisecond, tries to acquire the lock, sleeps
+But in Python 2, the ``lock.acquire()`` method has no timeout. To wait with a timeout, a Python
+2 condition variable sleeps for a millisecond, tries to acquire the lock, sleeps
 twice as long, and tries again. This exponential backoff reaches a maximum
 sleep time of 50ms.
 
-If PyMongo calls the condition variable's "wait" method with a short timeout,
+If PyMongo calls the condition variable's ``wait()`` method with a short timeout,
 the exponential backoff is restarted frequently. Overall, the condition variable
-is not waking a few times a second, but hundreds of times. (See `PYTHON-983`_.)
+is not waking a few times a second, but hundreds of times.
+(See `PYTHON-983 <https://jira.mongodb.org/browse/PYTHON-983>`__.)
 
-Thus the current design of periodic executors is surprisingly simple: they
-do a simple ``time.sleep`` for a half-second, check if it is time to wake or
+Thus, the current design of periodic executors is surprisingly simple: they
+call the ``time.sleep()`` method to pause for a half-second, check if it is time to wake or
 terminate, and sleep again.
-
-.. _Server Discovery And Monitoring Spec: https://github.com/mongodb/specifications/blob/master/source/server-discovery-and-monitoring/server-monitoring.rst#requesting-an-immediate-check
-
-.. _PYTHON-863: https://jira.mongodb.org/browse/PYTHON-863
-
-.. _PYTHON-983: https://jira.mongodb.org/browse/PYTHON-983