Async and Concurrent Hy

In earlier chapters we fetched web pages one at a time with urllib.request and waited for complete replies from language models before printing a single word. That sequential style is easy to read, but it leaves the CPU idle while waiting for network sockets. In this chapter I show you how to write concurrent Hy programs using Python’s asyncio library, httpx, queues, and worker pools.

The code for this chapter is in the directory source_code_for_examples/async_hy.

What You Will Build

We build six small modules that progress from core asyncio syntax to a complete concurrent web scraper and indexing pipeline:

  1. basics.hy: Coroutines (defn :async and fn :async), structured concurrency with asyncio.TaskGroup, timeouts, except* exception group handling, async context managers (with :async), async generators, and async comprehensions (lfor :async).
  2. bounded_fetch.hy: An HTTP client built on httpx.AsyncClient that bounds in-flight requests with asyncio.Semaphore, paces requests with a rate limiter, retries transient 503/429 errors with exponential backoff, and tests cleanly offline with httpx.MockTransport.
  3. streaming_llm.hy: An async generator that streams newline-delimited JSON (NDJSON) chunks from Ollama’s /api/chat endpoint and Server-Sent Events (SSE) lines using httpx streaming and .aiter-lines.
  4. pipeline.hy: A three-stage producer/consumer pipeline (crawl -> extract -> index) connected by bounded asyncio.Queue instances that apply backpressure automatically.
  5. cpu_offload.hy: Helpers that offload blocking calls to threads via asyncio.to_thread and pure-Python CPU work to separate processes via ProcessPoolExecutor, including the setup needed so spawned worker processes can import .hy modules.
  6. async_scraper.hy: A concurrent port of the book’s webscraping example that compares sequential and bounded-concurrent execution times against a deterministic mock news server.
 1 +-----------------------------------------------------------------------+
 2 |                       asyncio Event Loop (main)                       |
 3 +-----------------------------------------------------------------------+
 4        |                         |                         |
 5        v                         v                         v
 6 +--------------+         +---------------+         +--------------------+
 7 | TaskGroup    |         | Streaming LLM |         | Executor Bridge    |
 8 | + Semaphore  |         | .stream POST  |         | asyncio.to_thread  |
 9 | + Retries    |         | .aiter_lines  |         | ProcessPoolExecutor|
10 +--------------+         +---------------+         +--------------------+
11        |                         |                         |
12        v                         v                         v
13 +-----------------------------------------------------------------------+
14 |            Bounded asyncio.Queue Pipeline (Backpressure)              |
15 |   [url-queue] ---> crawl-workers ---> [page-queue (maxsize=4)]        |
16 |               ---> extract-workers -> [doc-queue  (maxsize=4)]        |
17 |               ---> index-worker   ---> Inverted Keyword Index         |
18 +-----------------------------------------------------------------------+

Why It Matters

Modern AI and data-collection programs spend most of their wall-clock time waiting on external systems:

  • Scraping ten news pages sequentially with a 100 ms response time takes a full second, plus any retry delays when a server returns HTTP 503.
  • Calling three external tools for an agent one after another multiplies tool latency.
  • Waiting for an LLM to finish a 500-token answer before displaying anything makes an interactive command-line tool feel broken.

At the same time, launching 1,000 unbounded concurrent requests will exhaust file descriptors or trigger rate limits on the target server. Production async code needs structured task lifetimes, explicit concurrency limits, retries with backoff, bounded queues, and a clear split between I/O-bound tasks and CPU-bound tasks.

The Concurrency Map

Before writing code, choose the concurrency model that matches your bottleneck:

Workload Type Bottleneck Right Tool in Hy Why
Many HTTP requests, database queries, LLM streams Network I/O wait asyncio (defn :async, TaskGroup, httpx.AsyncClient) Single thread multiplexes thousands of sockets with minimal memory and no lock contention
Legacy synchronous I/O library (urllib, sqlite3, local file reads) Blocking system call asyncio.to-thread or ThreadPoolExecutor Keeps the asyncio event loop responsive by running the blocking call on a worker thread
C extensions that release the GIL (numpy, hashlib, compression) Native CPU compute ThreadPoolExecutor / asyncio.to-thread Threads run in parallel once the C function releases the GIL
Pure-Python / pure-Hy CPU loops (parsing, prime search, graph search) CPU bytecode execution under standard GIL ProcessPoolExecutor via loop.run-in-executor Separate OS processes each have their own interpreter and bypass the GIL
Pure-Python CPU work on a Python 3.13/3.14+ free-threaded build (python3.14t) Multi-core CPU without process IPC ThreadPoolExecutor (verify with sys._is_gil_enabled) When the GIL is disabled at runtime, threads execute Python bytecode in parallel

The Hy Angle

Hy maps every Python async construct directly onto existing Lisp forms using the :async keyword modifier instead of inventing a separate set of macro names:

Construct Hy 1.3.1 Form Compiled Python
Named coroutine (defn :async fetch [url] ...) async def fetch(url): ...
Anonymous coroutine (fn :async [x] ...) async def _hy_anon(x): ...
Await expression (await (fetch url)) await fetch(url)
Async context manager (with [:async c (Client)] ...) async with Client() as c: ...
Async loop (for [:async line lines] ...) async for line in lines: ...
Async comprehension (lfor :async x aiter (* x x)) [x * x async for x in aiter]
Exception group handler (except* [eg ValueError] ...) except* ValueError as eg: ...

Because :async is a modifier on defn, fn, with, for, lfor, gfor, sfor, and dfor, the structure of your code stays identical when you move from synchronous Hy to asynchronous Hy.

Python Equivalent (hy2py)

You can verify how Hy compiles async forms at any time by running uv run hy2py. Here is a comparison between our Hy code in basics.hy and the Python AST code that hy2py generates:

 1 ;; Hy (from basics.hy)
 2 (defn :async run-concurrent [items delay-seconds]
 3   (setv start (time.perf-counter))
 4   (setv tasks [])
 5   (with [:async tg (asyncio.TaskGroup)]
 6     (for [item items]
 7       (.append tasks (.create-task tg (simulated-io item delay-seconds)))))
 8   (setv results (lfor t tasks (.result t)))
 9   #(results (- (time.perf-counter) start)))
10 
11 (defn :async collect-even-squares [n [delay-seconds 0.002]]
12   (lfor :async x (count-stream n delay-seconds)
13         :if (= (% x 2) 0)
14         (* x x)))

Running uv run hy2py basics.hy produces the exact Python equivalent:

 1 async def run_concurrent(items, delay_seconds):
 2     start = time.perf_counter()
 3     tasks = []
 4     _hy_anon_1 = None
 5     async with asyncio.TaskGroup() as tg:
 6         for item in items:
 7             tasks.append(tg.create_task(simulated_io(item, delay_seconds)))
 8         _hy_anon_1 = None
 9     results = [t.result() for t in tasks]
10     return (results, time.perf_counter() - start)
11 
12 async def collect_even_squares(n, delay_seconds=0.002):
13     return [x * x async for x in count_stream(n, delay_seconds) if x % 2 == 0]

Run It

The project directory source_code_for_examples/async_hy is layout-complete with pyproject.toml, conftest.py, six modules, demo.hy, and test_async.hy:

 1 async_hy/
 2 ├── basics.hy          # defn :async, fn :async, TaskGroup, timeout, except*, async generators
 3 ├── bounded_fetch.hy   # httpx.AsyncClient, Semaphore, exponential backoff, rate limiter
 4 ├── streaming_llm.hy   # Ollama NDJSON and SSE streaming via httpx .stream / .aiter-lines
 5 ├── pipeline.hy        # Bounded asyncio.Queue pipeline: crawl -> extract -> index
 6 ├── cpu_offload.hy     # asyncio.to_thread, ThreadPoolExecutor, ProcessPoolExecutor, GIL check
 7 ├── async_scraper.hy   # Case study: sequential vs concurrent BeautifulSoup scraper
 8 ├── demo.hy            # Guided tour that runs all six modules and prints timings
 9 ├── test_async.hy      # Deterministic pytest-asyncio test suite (11 tests, offline)
10 ├── conftest.py        # pytest hook to collect test_*.hy files
11 └── pyproject.toml     # Dependencies: hy, httpx, beautifulsoup4, pytest, pytest-asyncio

Install the environment, run the test suite, and run the guided tour:

1 cd source_code_for_examples/async_hy
2 uv sync
3 uv run pytest -q
4 uv run hy demo.hy

Here is the real output captured from uv run hy demo.hy:

 1 === 1. Async Basics & Structured Concurrency ===
 2 Sequential (4 x 30ms): 124.3 ms
 3 Concurrent TaskGroup : 31.2 ms
 4 fn :async squares    : [4, 9, 25, 49]
 5 Timeout (fast)       : {'ok': 'ready'}
 6 Timeout (expired)    : {'error': 'timeout'}
 7 except* ValueErrors  : ['negative item: -2', 'negative item: -5']
 8 except* KeyErrors    : ["'missing item'"]
 9 lfor :async squares  : [0, 4, 16, 36] (sum 0..5 = 15, 32.3 ms)
10 
11 === 2. Bounded HTTP Client with Retries & Rate Limiting ===
12 Fetched items        : 6
13 Max in-flight (cap 3): 3
14 Attempts per path    : {'/item/1': 1, '/item/2': 3, '/item/3': 1, '/item/4': 1, '/item/5': 2, '/item/6': 1}
15 
16 === 3. Streaming LLM Output (NDJSON / Ollama) ===
17 Streamed text        : Hy streams NDJSON tokens with for :async.
18 Chunks received      : 6
19 Time to first token  : 9.4 ms
20 Live Ollama check    : Ollama not reachable (ReadTimeout); used MockTransport stream.
21 
22 === 4. Producer/Consumer Queue Pipeline (crawl -> extract -> index) ===
23 Indexed documents    : 8
24 Distinct index terms : 134
25   term 'datalog' -> ['https://news.example/news/ai-research']
26   term 'asyncio' -> ['https://news.example/news/lisp-ecosystem', 'https://news.example/news/python-runtime']
27   term 'shacl' -> ['https://news.example/news/knowledge-graphs']
28   term 'ndjson' -> ['https://news.example/news/open-models']
29   term 'backpressure' -> ['https://news.example/news/data-engineering']
30 
31 === 5. Threads, Processes & Free-Threading Check ===
32 Runtime status       : {'python_version': '3.14.7', 'gil_enabled': True, 'free_threaded': False}
33 to_thread SHA-256[0] : 47d0f73c9d255333
34 ProcessPool primes   : [1229, 1033] (total 2262 primes < 20,000)
35 
36 === 6. Case Study: Sequential vs Concurrent Web Scraper ===
37 Pages scraped        : 8 (simulated 40ms latency + 1 retry)
38 Sequential time      : 396.1 ms
39 Concurrent (sem=4)   : 128.8 ms
40 Measured speedup     : 3.08x (max in-flight: 4)
41 Sample extracted summary from /news/lisp-ecosystem:
42   Hy programmers tap directly into Python asyncio, httpx, and structured concurrency

How It Works

Let’s walk through each of the six modules in detail.

1. Core Async Forms and Structured Concurrency: basics.hy

A function defined with (defn :async ...) returns a coroutine object when called. To execute it from synchronous code at the top level of a script, pass the coroutine to (asyncio.run ...). Inside a coroutine, (await expr) suspends the current coroutine until expr completes, allowing the event loop to run other ready tasks.

Here are the core functions in basics.hy:

 1 (defn :async simulated-io [label delay-seconds]
 2   "Simulate a non-blocking I/O operation that takes delay-seconds."
 3   (await (asyncio.sleep delay-seconds))
 4   {"label" label "delay" delay-seconds})
 5 
 6 
 7 (defn :async run-sequential [items delay-seconds]
 8   "Run simulated-io one item at a time and return #(results elapsed-seconds)."
 9   (setv start (time.perf-counter))
10   (setv results [])
11   (for [item items]
12     (.append results (await (simulated-io item delay-seconds))))
13   #(results (- (time.perf-counter) start)))
14 
15 
16 (defn :async run-concurrent [items delay-seconds]
17   "Run simulated-io concurrently with asyncio.TaskGroup and return #(results elapsed-seconds)."
18   (setv start (time.perf-counter))
19   (setv tasks [])
20   (with [:async tg (asyncio.TaskGroup)]
21     (for [item items]
22       (.append tasks (.create-task tg (simulated-io item delay-seconds)))))
23   (setv results (lfor t tasks (.result t)))
24   #(results (- (time.perf-counter) start)))
25 
26 
27 (defn :async run-with-timeout [coro timeout-seconds]
28   "Run coro inside asyncio.timeout; return {:ok result} or {:error 'timeout'}."
29   (try
30     (with [:async (asyncio.timeout timeout-seconds)]
31       {"ok" (await coro)})
32     (except [TimeoutError]
33       {"error" "timeout"})))
34 
35 
36 (defn :async run-taskgroup-with-except-star [items]
37   "Run risky-worker over items in a TaskGroup and classify failures with except*."
38   (setv value-errors []
39         key-errors []
40         tasks [])
41   (try
42     (with [:async tg (asyncio.TaskGroup)]
43       (for [item items]
44         (.append tasks (.create-task tg (risky-worker item)))))
45     (except* [eg ValueError]
46       (for [exc eg.exceptions]
47         (.append value-errors (str exc))))
48     (except* [eg KeyError]
49       (for [exc eg.exceptions]
50         (.append key-errors (str exc)))))
51   {"results" (if (or value-errors key-errors)
52                  []
53                  (lfor t tasks (.result t)))
54    "value_errors" value-errors
55    "key_errors" key-errors})
56 
57 
58 (defclass AsyncTimer []
59   "Async context manager that records elapsed wall-clock time in seconds."
60 
61   (defn __init__ [self label]
62     (setv self.label label
63           self.start-time 0.0
64           self.elapsed 0.0))
65 
66   (defn :async __aenter__ [self]
67     (setv self.start-time (time.perf-counter))
68     self)
69 
70   (defn :async __aexit__ [self exc-type exc-val exc-tb]
71     (await (asyncio.sleep 0))
72     (setv self.elapsed (- (time.perf-counter) self.start-time))
73     False))
74 
75 
76 (defn :async count-stream [n [delay-seconds 0.005]]
77   "Async generator yielding integers 0 .. n-1 after a short delay."
78   (for [i (range n)]
79     (await (asyncio.sleep delay-seconds))
80     (yield i)))
81 
82 
83 (defn :async collect-even-squares [n [delay-seconds 0.002]]
84   "Use an async list comprehension (lfor :async ...) over an async generator."
85   (lfor :async x (count-stream n delay-seconds)
86         :if (= (% x 2) 0)
87         (* x x)))

Four details in basics.hy are worth highlighting:

  1. Structured concurrency with TaskGroup: In run-concurrent, (with [:async tg (asyncio.TaskGroup)] ...) guarantees that every task spawned with (.create-task tg ...) has finished (or been cancelled and awaited) before execution leaves the with block. Unlike raw asyncio.create_task, TaskGroup never leaks orphan background tasks.
  2. Cancellation and asyncio.timeout: In run-with-timeout, (with [:async (asyncio.timeout timeout-seconds)] ...) cancels the inner task if the deadline expires and raises TimeoutError at the with boundary.
  3. Exception groups and except*: Because multiple tasks inside a TaskGroup can fail during the same turn of the event loop, TaskGroup wraps uncaught task exceptions in an ExceptionGroup. Hy supports Python’s except* syntax directly: (except* [eg ValueError] ...) catches the subgroup of ValueError exceptions while allowing a second (except* [eg KeyError] ...) clause in the same try form to handle KeyError failures from sibling tasks.
  4. Async context managers, generators, and comprehensions: Defining (defn :async __aenter__ [self] ...) and (defn :async __aexit__ [self exc-type exc-val exc-tb] ...) makes AsyncTimer work with (with [:async timer (AsyncTimer ...)] ...). Putting (yield i) inside a (defn :async ...) function turns count-stream into an async generator, which we consume with (lfor :async x ...) or (for [:async x ...] ...).

2. Bounded HTTP Concurrency, Retries, and Rate Limiting: bounded_fetch.hy

Spawning a task per URL in a TaskGroup without a concurrency limit will fire all requests simultaneously. In bounded_fetch.hy, we combine httpx.AsyncClient with asyncio.Semaphore, an interval rate limiter, and exponential backoff:

 1 (import asyncio)
 2 (import time)
 3 (import httpx)
 4 
 5 
 6 (setv RETRIABLE-STATUS-CODES #{429 500 502 503 504})
 7 
 8 
 9 (defclass AsyncRateLimiter []
10   "Enforce a minimum interval between request starts across concurrent tasks."
11 
12   (defn __init__ [self rate-per-second]
13     (setv self.min-interval (/ 1.0 rate-per-second)
14           self.lock (asyncio.Lock)
15           self.next-allowed 0.0))
16 
17   (defn :async acquire [self]
18     (with [:async self.lock]
19       (setv now (time.monotonic))
20       (when (< now self.next-allowed)
21         (await (asyncio.sleep (- self.next-allowed now)))
22         (setv now (time.monotonic)))
23       (setv self.next-allowed (+ now self.min-interval)))))
24 
25 
26 (defn :async fetch-with-retry [client url
27                                [max-retries 3]
28                                [base-delay 0.02]
29                                [semaphore None]
30                                [rate-limiter None]]
31   "Fetch url with optional semaphore, rate limiter, and exponential backoff."
32   (defn :async _do-attempt []
33     (when (is-not rate-limiter None)
34       (await (.acquire rate-limiter)))
35     (await (.get client url)))
36 
37   (setv attempt 0)
38   (while True
39     (+= attempt 1)
40     (try
41       (setv response
42             (if (is-not semaphore None)
43                 (with [:async semaphore]
44                   (await (_do-attempt)))
45                 (await (_do-attempt))))
46       (if (and (in response.status-code RETRIABLE-STATUS-CODES)
47                (<= attempt max-retries))
48           (await (asyncio.sleep (* base-delay (** 2 (- attempt 1)))))
49           (do
50             (.raise-for-status response)
51             (return {"url" url
52                      "status" response.status-code
53                      "text" response.text
54                      "attempts" attempt})))
55       (except [exc httpx.RequestError]
56         (when (> attempt max-retries)
57           (raise exc))
58         (await (asyncio.sleep (* base-delay (** 2 (- attempt 1)))))))))
59 
60 
61 (defn :async fetch-all-bounded [client urls
62                                 [max-concurrency 4]
63                                 [max-retries 3]
64                                 [base-delay 0.02]
65                                 [rate-per-second None]]
66   "Fetch all urls concurrently under a semaphore and optional rate limiter."
67   (setv sem (asyncio.Semaphore max-concurrency)
68         limiter (if (is-not rate-per-second None)
69                     (AsyncRateLimiter rate-per-second)
70                     None)
71         tasks [])
72   (with [:async tg (asyncio.TaskGroup)]
73     (for [url urls]
74       (.append tasks
75                (.create-task tg
76                  (fetch-with-retry client url
77                                    :max-retries max-retries
78                                    :base-delay base-delay
79                                    :semaphore sem
80                                    :rate-limiter limiter)))))
81   (lfor t tasks (.result t)))

Notice where the semaphore is acquired in fetch-with-retry:

  • (with [:async semaphore] (await (_do-attempt))) holds a semaphore slot only while the HTTP request is in flight, and releases the slot before (await (asyncio.sleep ...)) during a retry backoff delay. If a task slept while holding the semaphore, backing-off tasks would block healthy requests from using the connection pool.
  • AsyncRateLimiter uses an asyncio.Lock and time.monotonic to space request starts by 1.0 / rate-per-second seconds.

3. Streaming LLM Output (NDJSON and SSE): streaming_llm.hy

Local model servers such as Ollama stream responses as newline-delimited JSON (application/x-ndjson), where each line is a self-contained JSON object with a "message" chunk and a boolean "done" flag. OpenAI-compatible APIs use Server-Sent Events (text/event-stream), where each payload line starts with data: and the stream ends with data: [DONE].

In streaming_llm.hy, we parse both formats and expose an async generator stream-ollama-chat using (.stream client "POST" ...) and (.aiter-lines response):

 1 (defn parse-ndjson-line [line]
 2   "Parse one NDJSON line from an Ollama /api/chat stream.
 3   Return #(token is-done) or #(None False) for blank lines."
 4   (setv stripped (.strip line))
 5   (when (not stripped)
 6     (return #(None False)))
 7   (setv payload (json.loads stripped)
 8         msg (.get payload "message" {})
 9         token (.get msg "content" "")
10         done (bool (.get payload "done" False)))
11   #(token done))
12 
13 
14 (defn parse-sse-line [line]
15   "Parse one Server-Sent Events line ('data: ...').
16   Return #(token is-done) or #(None False) for comments/non-data lines."
17   (setv stripped (.strip line))
18   (when (or (not stripped) (not (.startswith stripped "data:")))
19     (return #(None False)))
20   (setv data-str (.strip (cut stripped 5 None)))
21   (when (= data-str "[DONE]")
22     (return #("" True)))
23   (setv payload (json.loads data-str)
24         choices (.get payload "choices" [])
25         delta (if choices (.get (get choices 0) "delta" {}) {})
26         token (.get delta "content" ""))
27   #(token False))
28 
29 
30 (defn :async stream-ollama-chat [client prompt
31                                  [model "qwen3.5:4b"]
32                                  [url "http://127.0.0.1:11434/api/chat"]]
33   "Async generator yielding text chunks from an Ollama /api/chat NDJSON stream."
34   (setv body {"model" model
35               "messages" [{"role" "user" "content" prompt}]
36               "stream" True})
37   (with [:async response (.stream client "POST" url :json body)]
38     (.raise-for-status response)
39     (for [:async line (.aiter-lines response)]
40       (setv [token done] (parse-ndjson-line line))
41       (when token
42         (yield token))
43       (when done
44         (break)))))

Because stream-ollama-chat is an async generator, callers process tokens as they arrive ((for [:async token (stream-ollama-chat ...)] ...)) and can measure time-to-first-token (TTFT) separately from total generation time. When Ollama is not running locally, try-live-ollama catches the connection or timeout error and reports that the offline FakeNDJSONByteStream transport was used.

4. Producer/Consumer Pipelines with asyncio.Queue: pipeline.hy

When a workflow has distinct stages with different speeds, such as downloading HTML, parsing DOM trees, and updating an index, connecting worker pools with bounded asyncio.Queue instances decouples the stages and bounds memory usage. If the parser falls behind the downloader, (await (.put page-queue ...)) pauses the crawl workers as soon as page-queue reaches maxsize.

Here is pipeline.hy:

  1 (import asyncio)
  2 (import re)
  3 (import bs4 [BeautifulSoup])
  4 
  5 
  6 (setv WORD-RE (re.compile r"[a-zA-Z]{3,}"))
  7 
  8 
  9 (defn extract-document [url html]
 10   "Parse HTML with BeautifulSoup and return title, links, and lowercase words."
 11   (setv soup (BeautifulSoup html "html.parser")
 12         title-tag (.find soup "title")
 13         title (if title-tag (.strip (.get-text title-tag)) "")
 14         links (lfor a (.find-all soup "a")
 15                     :if (.get a "href")
 16                     (.get a "href"))
 17         body-text (.get-text soup " " :strip True)
 18         words (lfor m (.findall WORD-RE body-text) (.lower m)))
 19   {"url" url
 20    "title" title
 21    "links" links
 22    "words" words})
 23 
 24 
 25 (defn :async run-crawl-extract-index [urls fetch-fn
 26                                       [crawl-workers 3]
 27                                       [extract-workers 2]
 28                                       [queue-maxsize 4]]
 29   "Run a three-stage asyncio.Queue pipeline (crawl -> extract -> index)
 30   with bounded queues for backpressure. Return the inverted index and docs."
 31   (setv url-queue (asyncio.Queue)
 32         page-queue (asyncio.Queue :maxsize queue-maxsize)
 33         doc-queue (asyncio.Queue :maxsize queue-maxsize)
 34         documents {}
 35         inverted-index {})
 36 
 37   (for [u urls]
 38     (.put-nowait url-queue u))
 39   (for [_ (range crawl-workers)]
 40     (.put-nowait url-queue None))
 41 
 42   (defn :async crawl-worker []
 43     (while True
 44       (setv url (await (.get url-queue)))
 45       (try
 46         (when (is url None)
 47           (break))
 48         (setv html (await (fetch-fn url)))
 49         (await (.put page-queue #(url html)))
 50         (finally
 51           (.task-done url-queue)))))
 52 
 53   (defn :async extract-worker []
 54     (while True
 55       (setv item (await (.get page-queue)))
 56       (try
 57         (when (is item None)
 58           (break))
 59         (setv [url html] item
 60               doc (extract-document url html))
 61         (await (.put doc-queue doc))
 62         (finally
 63           (.task-done page-queue)))))
 64 
 65   (defn :async index-worker []
 66     (while True
 67       (setv doc (await (.get doc-queue)))
 68       (try
 69         (when (is doc None)
 70           (break))
 71         (setv url (get doc "url"))
 72         (setv (get documents url) doc)
 73         (for [word (set (get doc "words"))]
 74           (when (not (in word inverted-index))
 75             (setv (get inverted-index word) (set)))
 76           (.add (get inverted-index word) url))
 77         (finally
 78           (.task-done doc-queue)))))
 79 
 80   ;; Stage coordination: when all crawl workers finish, send sentinel None
 81   ;; tokens to extract workers; when extract workers finish, stop index-worker.
 82   (defn :async coordinator []
 83     (with [:async crawl-tg (asyncio.TaskGroup)]
 84       (for [_ (range crawl-workers)]
 85         (.create-task crawl-tg (crawl-worker))))
 86     (for [_ (range extract-workers)]
 87       (await (.put page-queue None))))
 88 
 89   (defn :async extract-stage []
 90     (with [:async extract-tg (asyncio.TaskGroup)]
 91       (for [_ (range extract-workers)]
 92         (.create-task extract-tg (extract-worker))))
 93     (await (.put doc-queue None)))
 94 
 95   (with [:async tg (asyncio.TaskGroup)]
 96     (.create-task tg (coordinator))
 97     (.create-task tg (extract-stage))
 98     (.create-task tg (index-worker)))
 99 
100   (setv sorted-index
101         (dfor [word url-set] (.items inverted-index)
102               word (sorted url-set)))
103   {"documents" documents
104    "index" sorted-index})

Each stage shuts down cleanly using sentinel None values:

  1. Before starting, we enqueue one None sentinel per crawl-worker at the tail of url-queue.
  2. coordinator runs all crawl-worker tasks inside an inner TaskGroup. Once that TaskGroup exits, every URL has been fetched and placed onto page-queue, so coordinator pushes one None sentinel per extract-worker onto page-queue.
  3. extract-stage runs all extract-worker tasks in its own TaskGroup and then pushes a single None sentinel onto doc-queue to stop index-worker.

5. Threads, Processes, and Free-Threaded Python: cpu_offload.hy

Never run a CPU-heavy function or a blocking synchronous library directly on the asyncio event loop thread; doing so freezes every other coroutine until the function returns. Instead, delegate blocking or GIL-releasing calls to a thread pool with (asyncio.to-thread fn #* args), and delegate pure-Python CPU work to a ProcessPoolExecutor via (.run-in-executor loop pool fn arg).

Running ProcessPoolExecutor from a .hy script involves a subtle interaction with Python’s spawn start method (the default on macOS, Windows, and Python 3.14+ on Linux):

  1. When you launch a program with uv run hy demo.hy, sys.executable points to the hy wrapper script rather than python, and __main__.__file__ points to demo.hy.
  2. When multiprocessing spawns a worker process, it runs sys.executable -c "from multiprocessing.spawn import spawn_main; ..." and tries to compile __main__.__file__ as Python syntax before running your pool initializer.
  3. Even after the worker starts Python, a fresh interpreter cannot import functions defined in .hy modules until import hy has registered Hy’s import hook.

Our helper make-hy-process-pool in cpu_offload.hy solves all three issues in twenty lines:

 1 (defn ensure-python-spawn-executable []
 2   "When invoked via the hy CLI, sys.executable points to the hy script and
 3   __main__.__file__ points to a .hy script that Python's spawn bootstrapper
 4   cannot compile as .py. Point multiprocessing at the sibling python binary
 5   and clear .hy __main__.__file__ so worker processes start cleanly."
 6   (setv exe (Path sys.executable))
 7   (when (in exe.name #{"hy" "hy.exe"})
 8     (for [candidate ["python" "python3" "python.exe"]]
 9       (setv py (.with-name exe candidate))
10       (when (.exists py)
11         (multiprocessing.set-executable (str py))
12         (break))))
13   (setv main-mod (.get sys.modules "__main__")
14         main-file (getattr main-mod "__file__" None))
15   (when (and (isinstance main-file str) (.endswith main-file ".hy"))
16     (setv main-mod.__file__ None
17           main-mod.__spec__ None)))
18 
19 
20 (defn make-hy-process-pool [[max-workers 2]]
21   "Create a ProcessPoolExecutor whose workers import hy on startup so they can
22   unpickle functions defined in .hy modules."
23   (ensure-python-spawn-executable)
24   (ProcessPoolExecutor :max-workers max-workers
25                        :initializer builtins.__import__
26                        :initargs #("hy")))
27 
28 
29 (defn :async offload-with-to-thread [payloads [iterations 25000]]
30   "Run blocking-digest-work across payloads concurrently via asyncio.to_thread."
31   (setv tasks [])
32   (with [:async tg (asyncio.TaskGroup)]
33     (for [p payloads]
34       (.append tasks
35                (.create-task tg (asyncio.to-thread blocking-digest-work p iterations)))))
36   (lfor t tasks (.result t)))
37 
38 
39 (defn :async offload-cpu-process-pool [ranges [max-workers 2]]
40   "Run pure-Python count-primes-in-range across multiple worker processes."
41   (setv loop (asyncio.get-running-loop))
42   (with [pool (make-hy-process-pool :max-workers max-workers)]
43     (setv futures
44           (lfor r ranges
45                 (.run-in-executor loop pool count-primes-in-range r)))
46     (list (await (asyncio.gather #* futures)))))

Passing :initializer builtins.__import__ :initargs #("hy") ensures every worker process imports hy at startup before unpickling any task function from cpu_offload.hy.

A Note on Free-Threaded Python (PEP 703)

Python 3.13 introduced an experimental free-threaded build (python3.13t / python3.14t) that disables the Global Interpreter Lock (GIL), and Hy 1.3.1 runs on Python 3.9 through 3.15. You can check at runtime whether the current interpreter has the GIL enabled with python-gil-status in cpu_offload.hy:

1 (defn python-gil-status []
2   "Return a dict describing the current Python version and whether the GIL is enabled."
3   (setv check-fn (getattr sys "_is_gil_enabled" None)
4         gil-enabled (if (callable check-fn) (check-fn) True))
5   {"python_version" (get (.split sys.version) 0)
6    "gil_enabled" gil-enabled
7    "free_threaded" (not gil-enabled)})

Even on a free-threaded build, asyncio remains the right choice for high-concurrency network I/O because coroutines cost a few kilobytes of memory compared to OS threads and avoid shared-state data races by switching only at explicit await points.

6. Case Study: Porting the Web Scraper to Async (async_scraper.hy)

In the “Responsible Web Scraping” chapter (source_code_for_examples/webscraping), get_web_page.hy, get_page_data.hy, and npr_front_page_summary.hy used synchronous urllib.request.urlopen calls to download pages one by one and extract BeautifulSoup tag dictionaries and long headlines (anchor text longer than 40 characters).

In async_scraper.hy, we keep the same data extraction functions (get-element-data and parse-page-elements) and implement both a sequential scraper and a semaphore-bounded concurrent scraper:

 1 (defn get-element-data [element]
 2   "Extract text, tag name, class, and href from a BeautifulSoup tag.
 3   Direct port of get-element-data from source_code_for_examples/webscraping/get_page_data.hy."
 4   {"text" (.strip (.get-text element))
 5    "name" element.name
 6    "class" (.get element "class")
 7    "href" (.get element "href")})
 8 
 9 
10 (defn parse-page-elements [html]
11   "Parse HTML into structured title, a, h1, and h2 element dictionaries,
12   plus a headline summary (links with text > 40 chars, matching npr_front_page_summary.hy)."
13   (setv soup (BeautifulSoup html "html.parser")
14         titles (lfor e (.find-all soup "title") (get-element-data e))
15         anchors (lfor e (.find-all soup "a") (get-element-data e))
16         h1s (lfor e (.find-all soup "h1") (get-element-data e))
17         h2s (lfor e (.find-all soup "h2") (get-element-data e))
18         long-headlines (lfor a anchors
19                              :if (> (len (get a "text")) 40)
20                              (get a "text")))
21   {"title" (if titles (get (get titles 0) "text") "")
22    "a" anchors
23    "h1" h1s
24    "h2" h2s
25    "summary" (.join "\n\n" long-headlines)})
26 
27 
28 (defn :async scrape-pages-sequential [client urls]
29   "Fetch and parse each URL one at a time (sequential baseline).
30   Return #(pages-list elapsed-seconds)."
31   (setv start (time.perf-counter)
32         results [])
33   (for [url urls]
34     (setv resp (await (fetch-with-retry client url))
35           parsed (parse-page-elements (get resp "text")))
36     (.append results {"url" url
37                       "status" (get resp "status")
38                       "attempts" (get resp "attempts")
39                       #** parsed}))
40   #(results (- (time.perf-counter) start)))
41 
42 
43 (defn :async scrape-pages-concurrent [client urls [max-concurrency 4]]
44   "Fetch and parse all URLs concurrently using TaskGroup and a Semaphore.
45   Return #(pages-list elapsed-seconds)."
46   (setv start (time.perf-counter)
47         sem (asyncio.Semaphore max-concurrency)
48         tasks [])
49 
50   (defn :async _scrape-one [url]
51     (setv resp (await (fetch-with-retry client url :semaphore sem))
52           parsed (parse-page-elements (get resp "text")))
53     {"url" url
54      "status" (get resp "status")
55      "attempts" (get resp "attempts")
56      #** parsed})
57 
58   (with [:async tg (asyncio.TaskGroup)]
59     (for [url urls]
60       (.append tasks (.create-task tg (_scrape-one url)))))
61   (setv results (lfor t tasks (.result t)))
62   #(results (- (time.perf-counter) start)))

When we run compare-sequential-vs-concurrent over our 8 sample news pages with a 40 ms per-request simulated latency and one transient HTTP 503 failure on /news/open-models:

Mode In-Flight Cap Total Requests (with 1 Retry) Measured Time Speedup
scrape-pages-sequential 1 9 396.1 ms 1.00x
scrape-pages-concurrent 4 (Semaphore(4)) 9 128.8 ms 3.08x

Both functions return the exact same list of parsed page dictionaries in the same URL order, because scrape-pages-concurrent collects (.result t) from the tasks list in the order the tasks were created.

Tests

Testing async code against live websites or live LLM servers leads to slow, flaky tests. In test_async.hy, we configure pytest-asyncio with asyncio_mode = "auto" in pyproject.toml so every (defn :async test-... []) function runs automatically inside an event loop, and we replace network sockets with httpx.MockTransport.

Here is how make-mock-transport in bounded_fetch.hy simulates latency, tracks maximum in-flight concurrency, and injects a configurable number of initial HTTP 503 failures per path:

 1 (defn make-mock-transport [routes [latency-seconds 0.02] [flaky-counts None]]
 2   "Build an httpx.MockTransport with artificial latency and optional transient 503s.
 3   routes maps a URL path (like '/page/1') to a body string or #(status body).
 4   flaky-counts maps a URL path to the number of initial 503 responses before 200."
 5   (setv remaining-failures (dict (or flaky-counts {}))
 6         call-counts {}
 7         in-flight 0
 8         max-in-flight 0)
 9 
10   (defn :async handler [request]
11     (nonlocal in-flight max-in-flight)
12     (setv path request.url.path)
13     (setv (get call-counts path) (+ (.get call-counts path 0) 1))
14     (+= in-flight 1)
15     (when (> in-flight max-in-flight)
16       (setv max-in-flight in-flight))
17     (try
18       (when (> latency-seconds 0)
19         (await (asyncio.sleep latency-seconds)))
20       (setv fails-left (.get remaining-failures path 0))
21       (when (> fails-left 0)
22         (setv (get remaining-failures path) (- fails-left 1))
23         (return (httpx.Response 503 :text "Service Temporarily Unavailable")))
24       (when (not (in path routes))
25         (return (httpx.Response 404 :text "Not Found")))
26       (setv entry (get routes path))
27       (if (isinstance entry tuple)
28           (httpx.Response (get entry 0) :text (get entry 1))
29           (httpx.Response 200 :text entry))
30       (finally
31         (-= in-flight 1))))
32 
33   (setv transport (httpx.MockTransport handler))
34   {"transport" transport
35    "call_counts" call-counts
36    "stats" (fn [] {"max_in_flight" max-in-flight
37                    "call_counts" (dict call-counts)})})

Notice the (nonlocal in-flight max-in-flight) declaration inside handler. Each time a request enters handler, it increments in-flight, updates max-in-flight, sleeps for latency-seconds, and decrements in-flight inside (finally ...). Our test in test_async.hy then asserts both that retries succeeded and that the semaphore never allowed more than 3 concurrent requests:

 1 (defn :async test-bounded-fetch-semaphore-and-retries []
 2   (setv routes (dfor i (range 1 9) f"/p/{i}" f"body-{i}")
 3         mock (make-mock-transport routes
 4                                   :latency-seconds 0.015
 5                                   :flaky-counts {"/p/2" 2 "/p/6" 1})
 6         urls (lfor i (range 1 9) f"https://test.local/p/{i}"))
 7   (with [:async client (httpx.AsyncClient :transport (get mock "transport"))]
 8     (setv res (await (fetch-all-bounded client urls
 9                                         :max-concurrency 3
10                                         :max-retries 3
11                                         :base-delay 0.005))))
12   (assert (= (len res) 8))
13   (setv stats ((get mock "stats")))
14   (assert (<= (get stats "max_in_flight") 3))
15   (assert (= (get (get stats "call_counts") "/p/2") 3))
16   (assert (= (get (get stats "call_counts") "/p/6") 2)))

Run the full test suite with pytest:

1 $ uv run pytest -q
2 ...........                                                              [100%]
3 11 passed in 0.80s

Wrap Up

Hy gives you full access to Python’s modern concurrency stack with clean, consistent syntax: :async modifiers on functions, loops, context managers, and comprehensions; asyncio.TaskGroup and except* for structured concurrency; httpx.AsyncClient for bounded HTTP and LLM streaming; asyncio.Queue for backpressure-aware pipelines; and to-thread / ProcessPoolExecutor when work must leave the event loop.

Optional Practice Problems

  1. Add Jitter and Retry-After Support to fetch-with-retry (Small): Modify fetch-with-retry in bounded_fetch.hy so that when a response has status 429 and includes a "Retry-After" header (in seconds), it sleeps for (float (get response.headers "Retry-After")) instead of the exponential backoff formula. When "Retry-After" is absent, add uniform random jitter between 0 and 0.005 seconds to the backoff delay so concurrent retries do not fire in lockstep. Add a unit test in test_async.hy using httpx.MockTransport.

  2. Add an Async Summarization Stage to the Queue Pipeline (Open-Ended): Extend run-crawl-extract-index in pipeline.hy with a fourth stage (summarize-worker) that reads extracted documents from doc-queue, calls collect-chat-stream from streaming_llm.hy (using make-fake-ollama-transport by default or live Ollama when a --live flag is set) to generate a one-sentence summary of each page, and stores the summary alongside the document in the index. Bound the summarization stage with a semaphore so at most two LLM streams run at the same time.

Further Reading