Note: Uses the Python 3.5+ async/await syntax
asyncio supports the use of
Executor objects found in
concurrent.futures for scheduling tasks asynchronously. Event loops have the function
run_in_executor() which takes an
Executor object, a
Callable, and the Callable's parameters.
Scheduling a task for an
import asyncio from concurrent.futures import ThreadPoolExecutor def func(a, b): # Do time intensive stuff... return a + b async def main(loop): executor = ThreadPoolExecutor() result = await loop.run_in_executor(executor, func, "Hello,", " world!") print(result) if __name__ == "__main__": loop = asyncio.get_event_loop() loop.run_until_complete(main(loop))
Each event loop also has a "default"
Executor slot that can be assigned to an
Executor. To assign an
Executor and schedule tasks from the loop you use the
import asyncio from concurrent.futures import ThreadPoolExecutor def func(a, b): # Do time intensive stuff... return a + b async def main(loop): # NOTE: Using `None` as the first parameter designates the `default` Executor. result = await loop.run_in_executor(None, func, "Hello,", " world!") print(result) if __name__ == "__main__": loop = asyncio.get_event_loop() loop.set_default_executor(ThreadPoolExecutor()) loop.run_until_complete(main(loop))
There are two main types of
ThreadPoolExecutor and the
ThreadPoolExecutor contains a pool of threads which can either be manually set to a specific number of threads through the constructor or defaults to the number of cores on the machine times 5. The
ThreadPoolExecutor uses the pool of threads to execute tasks assigned to it and is generally better at CPU-bound operations rather than I/O bound operations.
Contrast that to the
ProcessPoolExecutor which spawns a new process for each task assigned to it. The
ProcessPoolExecutor can only take tasks and parameters that are picklable. The most common non-picklable tasks are the methods of objects. If you must schedule an object's method as a task in an
Executor you must use a