Async and Concurrent Hy
In earlier chapters we fetched web pages one at a time with urllib.request and waited for complete replies from language models before printing a single word. That sequential style is easy to read, but it leaves the CPU idle while waiting for network sockets. In this chapter I show you how to write concurrent Hy programs using Python’s asyncio library, httpx, queues, and worker pools.
The code for this chapter is in the directory source_code_for_examples/async_hy.
What You Will Build
We build six small modules that progress from core asyncio syntax to a complete concurrent web scraper and indexing pipeline:
- basics.hy: Coroutines (
defn :asyncandfn :async), structured concurrency withasyncio.TaskGroup, timeouts,except*exception group handling, async context managers (with :async), async generators, and async comprehensions (lfor :async). - bounded_fetch.hy: An HTTP client built on
httpx.AsyncClientthat bounds in-flight requests withasyncio.Semaphore, paces requests with a rate limiter, retries transient503/429errors with exponential backoff, and tests cleanly offline withhttpx.MockTransport. - streaming_llm.hy: An async generator that streams newline-delimited JSON (NDJSON) chunks from Ollama’s
/api/chatendpoint and Server-Sent Events (SSE) lines usinghttpxstreaming and.aiter-lines. - pipeline.hy: A three-stage producer/consumer pipeline (
crawl -> extract -> index) connected by boundedasyncio.Queueinstances that apply backpressure automatically. - cpu_offload.hy: Helpers that offload blocking calls to threads via
asyncio.to_threadand pure-Python CPU work to separate processes viaProcessPoolExecutor, including the setup needed so spawned worker processes can import.hymodules. - async_scraper.hy: A concurrent port of the book’s
webscrapingexample that compares sequential and bounded-concurrent execution times against a deterministic mock news server.
1 +-----------------------------------------------------------------------+
2 | asyncio Event Loop (main) |
3 +-----------------------------------------------------------------------+
4 | | |
5 v v v
6 +--------------+ +---------------+ +--------------------+
7 | TaskGroup | | Streaming LLM | | Executor Bridge |
8 | + Semaphore | | .stream POST | | asyncio.to_thread |
9 | + Retries | | .aiter_lines | | ProcessPoolExecutor|
10 +--------------+ +---------------+ +--------------------+
11 | | |
12 v v v
13 +-----------------------------------------------------------------------+
14 | Bounded asyncio.Queue Pipeline (Backpressure) |
15 | [url-queue] ---> crawl-workers ---> [page-queue (maxsize=4)] |
16 | ---> extract-workers -> [doc-queue (maxsize=4)] |
17 | ---> index-worker ---> Inverted Keyword Index |
18 +-----------------------------------------------------------------------+
Why It Matters
Modern AI and data-collection programs spend most of their wall-clock time waiting on external systems:
- Scraping ten news pages sequentially with a 100 ms response time takes a full second, plus any retry delays when a server returns HTTP 503.
- Calling three external tools for an agent one after another multiplies tool latency.
- Waiting for an LLM to finish a 500-token answer before displaying anything makes an interactive command-line tool feel broken.
At the same time, launching 1,000 unbounded concurrent requests will exhaust file descriptors or trigger rate limits on the target server. Production async code needs structured task lifetimes, explicit concurrency limits, retries with backoff, bounded queues, and a clear split between I/O-bound tasks and CPU-bound tasks.
The Concurrency Map
Before writing code, choose the concurrency model that matches your bottleneck:
| Workload Type | Bottleneck | Right Tool in Hy | Why |
|---|---|---|---|
| Many HTTP requests, database queries, LLM streams | Network I/O wait |
asyncio (defn :async, TaskGroup, httpx.AsyncClient) |
Single thread multiplexes thousands of sockets with minimal memory and no lock contention |
Legacy synchronous I/O library (urllib, sqlite3, local file reads) |
Blocking system call |
asyncio.to-thread or ThreadPoolExecutor
|
Keeps the asyncio event loop responsive by running the blocking call on a worker thread |
C extensions that release the GIL (numpy, hashlib, compression) |
Native CPU compute |
ThreadPoolExecutor / asyncio.to-thread
|
Threads run in parallel once the C function releases the GIL |
| Pure-Python / pure-Hy CPU loops (parsing, prime search, graph search) | CPU bytecode execution under standard GIL |
ProcessPoolExecutor via loop.run-in-executor
|
Separate OS processes each have their own interpreter and bypass the GIL |
Pure-Python CPU work on a Python 3.13/3.14+ free-threaded build (python3.14t) |
Multi-core CPU without process IPC |
ThreadPoolExecutor (verify with sys._is_gil_enabled) |
When the GIL is disabled at runtime, threads execute Python bytecode in parallel |
The Hy Angle
Hy maps every Python async construct directly onto existing Lisp forms using the :async keyword modifier instead of inventing a separate set of macro names:
| Construct | Hy 1.3.1 Form | Compiled Python |
|---|---|---|
| Named coroutine | (defn :async fetch [url] ...) |
async def fetch(url): ... |
| Anonymous coroutine | (fn :async [x] ...) |
async def _hy_anon(x): ... |
| Await expression | (await (fetch url)) |
await fetch(url) |
| Async context manager | (with [:async c (Client)] ...) |
async with Client() as c: ... |
| Async loop | (for [:async line lines] ...) |
async for line in lines: ... |
| Async comprehension | (lfor :async x aiter (* x x)) |
[x * x async for x in aiter] |
| Exception group handler | (except* [eg ValueError] ...) |
except* ValueError as eg: ... |
Because :async is a modifier on defn, fn, with, for, lfor, gfor, sfor, and dfor, the structure of your code stays identical when you move from synchronous Hy to asynchronous Hy.
Python Equivalent (hy2py)
You can verify how Hy compiles async forms at any time by running uv run hy2py. Here is a comparison between our Hy code in basics.hy and the Python AST code that hy2py generates:
1 ;; Hy (from basics.hy)
2 (defn :async run-concurrent [items delay-seconds]
3 (setv start (time.perf-counter))
4 (setv tasks [])
5 (with [:async tg (asyncio.TaskGroup)]
6 (for [item items]
7 (.append tasks (.create-task tg (simulated-io item delay-seconds)))))
8 (setv results (lfor t tasks (.result t)))
9 #(results (- (time.perf-counter) start)))
10
11 (defn :async collect-even-squares [n [delay-seconds 0.002]]
12 (lfor :async x (count-stream n delay-seconds)
13 :if (= (% x 2) 0)
14 (* x x)))
Running uv run hy2py basics.hy produces the exact Python equivalent:
1 async def run_concurrent(items, delay_seconds):
2 start = time.perf_counter()
3 tasks = []
4 _hy_anon_1 = None
5 async with asyncio.TaskGroup() as tg:
6 for item in items:
7 tasks.append(tg.create_task(simulated_io(item, delay_seconds)))
8 _hy_anon_1 = None
9 results = [t.result() for t in tasks]
10 return (results, time.perf_counter() - start)
11
12 async def collect_even_squares(n, delay_seconds=0.002):
13 return [x * x async for x in count_stream(n, delay_seconds) if x % 2 == 0]
Run It
The project directory source_code_for_examples/async_hy is layout-complete with pyproject.toml, conftest.py, six modules, demo.hy, and test_async.hy:
1 async_hy/
2 ├── basics.hy # defn :async, fn :async, TaskGroup, timeout, except*, async generators
3 ├── bounded_fetch.hy # httpx.AsyncClient, Semaphore, exponential backoff, rate limiter
4 ├── streaming_llm.hy # Ollama NDJSON and SSE streaming via httpx .stream / .aiter-lines
5 ├── pipeline.hy # Bounded asyncio.Queue pipeline: crawl -> extract -> index
6 ├── cpu_offload.hy # asyncio.to_thread, ThreadPoolExecutor, ProcessPoolExecutor, GIL check
7 ├── async_scraper.hy # Case study: sequential vs concurrent BeautifulSoup scraper
8 ├── demo.hy # Guided tour that runs all six modules and prints timings
9 ├── test_async.hy # Deterministic pytest-asyncio test suite (11 tests, offline)
10 ├── conftest.py # pytest hook to collect test_*.hy files
11 └── pyproject.toml # Dependencies: hy, httpx, beautifulsoup4, pytest, pytest-asyncio
Install the environment, run the test suite, and run the guided tour:
1 cd source_code_for_examples/async_hy
2 uv sync
3 uv run pytest -q
4 uv run hy demo.hy
Here is the real output captured from uv run hy demo.hy:
1 === 1. Async Basics & Structured Concurrency ===
2 Sequential (4 x 30ms): 124.3 ms
3 Concurrent TaskGroup : 31.2 ms
4 fn :async squares : [4, 9, 25, 49]
5 Timeout (fast) : {'ok': 'ready'}
6 Timeout (expired) : {'error': 'timeout'}
7 except* ValueErrors : ['negative item: -2', 'negative item: -5']
8 except* KeyErrors : ["'missing item'"]
9 lfor :async squares : [0, 4, 16, 36] (sum 0..5 = 15, 32.3 ms)
10
11 === 2. Bounded HTTP Client with Retries & Rate Limiting ===
12 Fetched items : 6
13 Max in-flight (cap 3): 3
14 Attempts per path : {'/item/1': 1, '/item/2': 3, '/item/3': 1, '/item/4': 1, '/item/5': 2, '/item/6': 1}
15
16 === 3. Streaming LLM Output (NDJSON / Ollama) ===
17 Streamed text : Hy streams NDJSON tokens with for :async.
18 Chunks received : 6
19 Time to first token : 9.4 ms
20 Live Ollama check : Ollama not reachable (ReadTimeout); used MockTransport stream.
21
22 === 4. Producer/Consumer Queue Pipeline (crawl -> extract -> index) ===
23 Indexed documents : 8
24 Distinct index terms : 134
25 term 'datalog' -> ['https://news.example/news/ai-research']
26 term 'asyncio' -> ['https://news.example/news/lisp-ecosystem', 'https://news.example/news/python-runtime']
27 term 'shacl' -> ['https://news.example/news/knowledge-graphs']
28 term 'ndjson' -> ['https://news.example/news/open-models']
29 term 'backpressure' -> ['https://news.example/news/data-engineering']
30
31 === 5. Threads, Processes & Free-Threading Check ===
32 Runtime status : {'python_version': '3.14.7', 'gil_enabled': True, 'free_threaded': False}
33 to_thread SHA-256[0] : 47d0f73c9d255333
34 ProcessPool primes : [1229, 1033] (total 2262 primes < 20,000)
35
36 === 6. Case Study: Sequential vs Concurrent Web Scraper ===
37 Pages scraped : 8 (simulated 40ms latency + 1 retry)
38 Sequential time : 396.1 ms
39 Concurrent (sem=4) : 128.8 ms
40 Measured speedup : 3.08x (max in-flight: 4)
41 Sample extracted summary from /news/lisp-ecosystem:
42 Hy programmers tap directly into Python asyncio, httpx, and structured concurrency
How It Works
Let’s walk through each of the six modules in detail.
1. Core Async Forms and Structured Concurrency: basics.hy
A function defined with (defn :async ...) returns a coroutine object when called. To execute it from synchronous code at the top level of a script, pass the coroutine to (asyncio.run ...). Inside a coroutine, (await expr) suspends the current coroutine until expr completes, allowing the event loop to run other ready tasks.
Here are the core functions in basics.hy:
1 (defn :async simulated-io [label delay-seconds]
2 "Simulate a non-blocking I/O operation that takes delay-seconds."
3 (await (asyncio.sleep delay-seconds))
4 {"label" label "delay" delay-seconds})
5
6
7 (defn :async run-sequential [items delay-seconds]
8 "Run simulated-io one item at a time and return #(results elapsed-seconds)."
9 (setv start (time.perf-counter))
10 (setv results [])
11 (for [item items]
12 (.append results (await (simulated-io item delay-seconds))))
13 #(results (- (time.perf-counter) start)))
14
15
16 (defn :async run-concurrent [items delay-seconds]
17 "Run simulated-io concurrently with asyncio.TaskGroup and return #(results elapsed-seconds)."
18 (setv start (time.perf-counter))
19 (setv tasks [])
20 (with [:async tg (asyncio.TaskGroup)]
21 (for [item items]
22 (.append tasks (.create-task tg (simulated-io item delay-seconds)))))
23 (setv results (lfor t tasks (.result t)))
24 #(results (- (time.perf-counter) start)))
25
26
27 (defn :async run-with-timeout [coro timeout-seconds]
28 "Run coro inside asyncio.timeout; return {:ok result} or {:error 'timeout'}."
29 (try
30 (with [:async (asyncio.timeout timeout-seconds)]
31 {"ok" (await coro)})
32 (except [TimeoutError]
33 {"error" "timeout"})))
34
35
36 (defn :async run-taskgroup-with-except-star [items]
37 "Run risky-worker over items in a TaskGroup and classify failures with except*."
38 (setv value-errors []
39 key-errors []
40 tasks [])
41 (try
42 (with [:async tg (asyncio.TaskGroup)]
43 (for [item items]
44 (.append tasks (.create-task tg (risky-worker item)))))
45 (except* [eg ValueError]
46 (for [exc eg.exceptions]
47 (.append value-errors (str exc))))
48 (except* [eg KeyError]
49 (for [exc eg.exceptions]
50 (.append key-errors (str exc)))))
51 {"results" (if (or value-errors key-errors)
52 []
53 (lfor t tasks (.result t)))
54 "value_errors" value-errors
55 "key_errors" key-errors})
56
57
58 (defclass AsyncTimer []
59 "Async context manager that records elapsed wall-clock time in seconds."
60
61 (defn __init__ [self label]
62 (setv self.label label
63 self.start-time 0.0
64 self.elapsed 0.0))
65
66 (defn :async __aenter__ [self]
67 (setv self.start-time (time.perf-counter))
68 self)
69
70 (defn :async __aexit__ [self exc-type exc-val exc-tb]
71 (await (asyncio.sleep 0))
72 (setv self.elapsed (- (time.perf-counter) self.start-time))
73 False))
74
75
76 (defn :async count-stream [n [delay-seconds 0.005]]
77 "Async generator yielding integers 0 .. n-1 after a short delay."
78 (for [i (range n)]
79 (await (asyncio.sleep delay-seconds))
80 (yield i)))
81
82
83 (defn :async collect-even-squares [n [delay-seconds 0.002]]
84 "Use an async list comprehension (lfor :async ...) over an async generator."
85 (lfor :async x (count-stream n delay-seconds)
86 :if (= (% x 2) 0)
87 (* x x)))
Four details in basics.hy are worth highlighting:
- Structured concurrency with
TaskGroup: Inrun-concurrent,(with [:async tg (asyncio.TaskGroup)] ...)guarantees that every task spawned with(.create-task tg ...)has finished (or been cancelled and awaited) before execution leaves thewithblock. Unlike rawasyncio.create_task,TaskGroupnever leaks orphan background tasks. - Cancellation and
asyncio.timeout: Inrun-with-timeout,(with [:async (asyncio.timeout timeout-seconds)] ...)cancels the inner task if the deadline expires and raisesTimeoutErrorat thewithboundary. - Exception groups and
except*: Because multiple tasks inside aTaskGroupcan fail during the same turn of the event loop,TaskGroupwraps uncaught task exceptions in anExceptionGroup. Hy supports Python’sexcept*syntax directly:(except* [eg ValueError] ...)catches the subgroup ofValueErrorexceptions while allowing a second(except* [eg KeyError] ...)clause in the sametryform to handleKeyErrorfailures from sibling tasks. - Async context managers, generators, and comprehensions: Defining
(defn :async __aenter__ [self] ...)and(defn :async __aexit__ [self exc-type exc-val exc-tb] ...)makesAsyncTimerwork with(with [:async timer (AsyncTimer ...)] ...). Putting(yield i)inside a(defn :async ...)function turnscount-streaminto an async generator, which we consume with(lfor :async x ...)or(for [:async x ...] ...).
2. Bounded HTTP Concurrency, Retries, and Rate Limiting: bounded_fetch.hy
Spawning a task per URL in a TaskGroup without a concurrency limit will fire all requests simultaneously. In bounded_fetch.hy, we combine httpx.AsyncClient with asyncio.Semaphore, an interval rate limiter, and exponential backoff:
1 (import asyncio)
2 (import time)
3 (import httpx)
4
5
6 (setv RETRIABLE-STATUS-CODES #{429 500 502 503 504})
7
8
9 (defclass AsyncRateLimiter []
10 "Enforce a minimum interval between request starts across concurrent tasks."
11
12 (defn __init__ [self rate-per-second]
13 (setv self.min-interval (/ 1.0 rate-per-second)
14 self.lock (asyncio.Lock)
15 self.next-allowed 0.0))
16
17 (defn :async acquire [self]
18 (with [:async self.lock]
19 (setv now (time.monotonic))
20 (when (< now self.next-allowed)
21 (await (asyncio.sleep (- self.next-allowed now)))
22 (setv now (time.monotonic)))
23 (setv self.next-allowed (+ now self.min-interval)))))
24
25
26 (defn :async fetch-with-retry [client url
27 [max-retries 3]
28 [base-delay 0.02]
29 [semaphore None]
30 [rate-limiter None]]
31 "Fetch url with optional semaphore, rate limiter, and exponential backoff."
32 (defn :async _do-attempt []
33 (when (is-not rate-limiter None)
34 (await (.acquire rate-limiter)))
35 (await (.get client url)))
36
37 (setv attempt 0)
38 (while True
39 (+= attempt 1)
40 (try
41 (setv response
42 (if (is-not semaphore None)
43 (with [:async semaphore]
44 (await (_do-attempt)))
45 (await (_do-attempt))))
46 (if (and (in response.status-code RETRIABLE-STATUS-CODES)
47 (<= attempt max-retries))
48 (await (asyncio.sleep (* base-delay (** 2 (- attempt 1)))))
49 (do
50 (.raise-for-status response)
51 (return {"url" url
52 "status" response.status-code
53 "text" response.text
54 "attempts" attempt})))
55 (except [exc httpx.RequestError]
56 (when (> attempt max-retries)
57 (raise exc))
58 (await (asyncio.sleep (* base-delay (** 2 (- attempt 1)))))))))
59
60
61 (defn :async fetch-all-bounded [client urls
62 [max-concurrency 4]
63 [max-retries 3]
64 [base-delay 0.02]
65 [rate-per-second None]]
66 "Fetch all urls concurrently under a semaphore and optional rate limiter."
67 (setv sem (asyncio.Semaphore max-concurrency)
68 limiter (if (is-not rate-per-second None)
69 (AsyncRateLimiter rate-per-second)
70 None)
71 tasks [])
72 (with [:async tg (asyncio.TaskGroup)]
73 (for [url urls]
74 (.append tasks
75 (.create-task tg
76 (fetch-with-retry client url
77 :max-retries max-retries
78 :base-delay base-delay
79 :semaphore sem
80 :rate-limiter limiter)))))
81 (lfor t tasks (.result t)))
Notice where the semaphore is acquired in fetch-with-retry:
(with [:async semaphore] (await (_do-attempt)))holds a semaphore slot only while the HTTP request is in flight, and releases the slot before(await (asyncio.sleep ...))during a retry backoff delay. If a task slept while holding the semaphore, backing-off tasks would block healthy requests from using the connection pool.AsyncRateLimiteruses anasyncio.Lockandtime.monotonicto space request starts by1.0 / rate-per-secondseconds.
3. Streaming LLM Output (NDJSON and SSE): streaming_llm.hy
Local model servers such as Ollama stream responses as newline-delimited JSON (application/x-ndjson), where each line is a self-contained JSON object with a "message" chunk and a boolean "done" flag. OpenAI-compatible APIs use Server-Sent Events (text/event-stream), where each payload line starts with data: and the stream ends with data: [DONE].
In streaming_llm.hy, we parse both formats and expose an async generator stream-ollama-chat using (.stream client "POST" ...) and (.aiter-lines response):
1 (defn parse-ndjson-line [line]
2 "Parse one NDJSON line from an Ollama /api/chat stream.
3 Return #(token is-done) or #(None False) for blank lines."
4 (setv stripped (.strip line))
5 (when (not stripped)
6 (return #(None False)))
7 (setv payload (json.loads stripped)
8 msg (.get payload "message" {})
9 token (.get msg "content" "")
10 done (bool (.get payload "done" False)))
11 #(token done))
12
13
14 (defn parse-sse-line [line]
15 "Parse one Server-Sent Events line ('data: ...').
16 Return #(token is-done) or #(None False) for comments/non-data lines."
17 (setv stripped (.strip line))
18 (when (or (not stripped) (not (.startswith stripped "data:")))
19 (return #(None False)))
20 (setv data-str (.strip (cut stripped 5 None)))
21 (when (= data-str "[DONE]")
22 (return #("" True)))
23 (setv payload (json.loads data-str)
24 choices (.get payload "choices" [])
25 delta (if choices (.get (get choices 0) "delta" {}) {})
26 token (.get delta "content" ""))
27 #(token False))
28
29
30 (defn :async stream-ollama-chat [client prompt
31 [model "qwen3.5:4b"]
32 [url "http://127.0.0.1:11434/api/chat"]]
33 "Async generator yielding text chunks from an Ollama /api/chat NDJSON stream."
34 (setv body {"model" model
35 "messages" [{"role" "user" "content" prompt}]
36 "stream" True})
37 (with [:async response (.stream client "POST" url :json body)]
38 (.raise-for-status response)
39 (for [:async line (.aiter-lines response)]
40 (setv [token done] (parse-ndjson-line line))
41 (when token
42 (yield token))
43 (when done
44 (break)))))
Because stream-ollama-chat is an async generator, callers process tokens as they arrive ((for [:async token (stream-ollama-chat ...)] ...)) and can measure time-to-first-token (TTFT) separately from total generation time. When Ollama is not running locally, try-live-ollama catches the connection or timeout error and reports that the offline FakeNDJSONByteStream transport was used.
4. Producer/Consumer Pipelines with asyncio.Queue: pipeline.hy
When a workflow has distinct stages with different speeds, such as downloading HTML, parsing DOM trees, and updating an index, connecting worker pools with bounded asyncio.Queue instances decouples the stages and bounds memory usage. If the parser falls behind the downloader, (await (.put page-queue ...)) pauses the crawl workers as soon as page-queue reaches maxsize.
Here is pipeline.hy:
1 (import asyncio)
2 (import re)
3 (import bs4 [BeautifulSoup])
4
5
6 (setv WORD-RE (re.compile r"[a-zA-Z]{3,}"))
7
8
9 (defn extract-document [url html]
10 "Parse HTML with BeautifulSoup and return title, links, and lowercase words."
11 (setv soup (BeautifulSoup html "html.parser")
12 title-tag (.find soup "title")
13 title (if title-tag (.strip (.get-text title-tag)) "")
14 links (lfor a (.find-all soup "a")
15 :if (.get a "href")
16 (.get a "href"))
17 body-text (.get-text soup " " :strip True)
18 words (lfor m (.findall WORD-RE body-text) (.lower m)))
19 {"url" url
20 "title" title
21 "links" links
22 "words" words})
23
24
25 (defn :async run-crawl-extract-index [urls fetch-fn
26 [crawl-workers 3]
27 [extract-workers 2]
28 [queue-maxsize 4]]
29 "Run a three-stage asyncio.Queue pipeline (crawl -> extract -> index)
30 with bounded queues for backpressure. Return the inverted index and docs."
31 (setv url-queue (asyncio.Queue)
32 page-queue (asyncio.Queue :maxsize queue-maxsize)
33 doc-queue (asyncio.Queue :maxsize queue-maxsize)
34 documents {}
35 inverted-index {})
36
37 (for [u urls]
38 (.put-nowait url-queue u))
39 (for [_ (range crawl-workers)]
40 (.put-nowait url-queue None))
41
42 (defn :async crawl-worker []
43 (while True
44 (setv url (await (.get url-queue)))
45 (try
46 (when (is url None)
47 (break))
48 (setv html (await (fetch-fn url)))
49 (await (.put page-queue #(url html)))
50 (finally
51 (.task-done url-queue)))))
52
53 (defn :async extract-worker []
54 (while True
55 (setv item (await (.get page-queue)))
56 (try
57 (when (is item None)
58 (break))
59 (setv [url html] item
60 doc (extract-document url html))
61 (await (.put doc-queue doc))
62 (finally
63 (.task-done page-queue)))))
64
65 (defn :async index-worker []
66 (while True
67 (setv doc (await (.get doc-queue)))
68 (try
69 (when (is doc None)
70 (break))
71 (setv url (get doc "url"))
72 (setv (get documents url) doc)
73 (for [word (set (get doc "words"))]
74 (when (not (in word inverted-index))
75 (setv (get inverted-index word) (set)))
76 (.add (get inverted-index word) url))
77 (finally
78 (.task-done doc-queue)))))
79
80 ;; Stage coordination: when all crawl workers finish, send sentinel None
81 ;; tokens to extract workers; when extract workers finish, stop index-worker.
82 (defn :async coordinator []
83 (with [:async crawl-tg (asyncio.TaskGroup)]
84 (for [_ (range crawl-workers)]
85 (.create-task crawl-tg (crawl-worker))))
86 (for [_ (range extract-workers)]
87 (await (.put page-queue None))))
88
89 (defn :async extract-stage []
90 (with [:async extract-tg (asyncio.TaskGroup)]
91 (for [_ (range extract-workers)]
92 (.create-task extract-tg (extract-worker))))
93 (await (.put doc-queue None)))
94
95 (with [:async tg (asyncio.TaskGroup)]
96 (.create-task tg (coordinator))
97 (.create-task tg (extract-stage))
98 (.create-task tg (index-worker)))
99
100 (setv sorted-index
101 (dfor [word url-set] (.items inverted-index)
102 word (sorted url-set)))
103 {"documents" documents
104 "index" sorted-index})
Each stage shuts down cleanly using sentinel None values:
- Before starting, we enqueue one
Nonesentinel percrawl-workerat the tail ofurl-queue. coordinatorruns allcrawl-workertasks inside an innerTaskGroup. Once thatTaskGroupexits, every URL has been fetched and placed ontopage-queue, socoordinatorpushes oneNonesentinel perextract-workerontopage-queue.extract-stageruns allextract-workertasks in its ownTaskGroupand then pushes a singleNonesentinel ontodoc-queueto stopindex-worker.
5. Threads, Processes, and Free-Threaded Python: cpu_offload.hy
Never run a CPU-heavy function or a blocking synchronous library directly on the asyncio event loop thread; doing so freezes every other coroutine until the function returns. Instead, delegate blocking or GIL-releasing calls to a thread pool with (asyncio.to-thread fn #* args), and delegate pure-Python CPU work to a ProcessPoolExecutor via (.run-in-executor loop pool fn arg).
Running ProcessPoolExecutor from a .hy script involves a subtle interaction with Python’s spawn start method (the default on macOS, Windows, and Python 3.14+ on Linux):
- When you launch a program with
uv run hy demo.hy,sys.executablepoints to thehywrapper script rather thanpython, and__main__.__file__points todemo.hy. - When
multiprocessingspawns a worker process, it runssys.executable -c "from multiprocessing.spawn import spawn_main; ..."and tries to compile__main__.__file__as Python syntax before running your pool initializer. - Even after the worker starts Python, a fresh interpreter cannot import functions defined in
.hymodules untilimport hyhas registered Hy’s import hook.
Our helper make-hy-process-pool in cpu_offload.hy solves all three issues in twenty lines:
1 (defn ensure-python-spawn-executable []
2 "When invoked via the hy CLI, sys.executable points to the hy script and
3 __main__.__file__ points to a .hy script that Python's spawn bootstrapper
4 cannot compile as .py. Point multiprocessing at the sibling python binary
5 and clear .hy __main__.__file__ so worker processes start cleanly."
6 (setv exe (Path sys.executable))
7 (when (in exe.name #{"hy" "hy.exe"})
8 (for [candidate ["python" "python3" "python.exe"]]
9 (setv py (.with-name exe candidate))
10 (when (.exists py)
11 (multiprocessing.set-executable (str py))
12 (break))))
13 (setv main-mod (.get sys.modules "__main__")
14 main-file (getattr main-mod "__file__" None))
15 (when (and (isinstance main-file str) (.endswith main-file ".hy"))
16 (setv main-mod.__file__ None
17 main-mod.__spec__ None)))
18
19
20 (defn make-hy-process-pool [[max-workers 2]]
21 "Create a ProcessPoolExecutor whose workers import hy on startup so they can
22 unpickle functions defined in .hy modules."
23 (ensure-python-spawn-executable)
24 (ProcessPoolExecutor :max-workers max-workers
25 :initializer builtins.__import__
26 :initargs #("hy")))
27
28
29 (defn :async offload-with-to-thread [payloads [iterations 25000]]
30 "Run blocking-digest-work across payloads concurrently via asyncio.to_thread."
31 (setv tasks [])
32 (with [:async tg (asyncio.TaskGroup)]
33 (for [p payloads]
34 (.append tasks
35 (.create-task tg (asyncio.to-thread blocking-digest-work p iterations)))))
36 (lfor t tasks (.result t)))
37
38
39 (defn :async offload-cpu-process-pool [ranges [max-workers 2]]
40 "Run pure-Python count-primes-in-range across multiple worker processes."
41 (setv loop (asyncio.get-running-loop))
42 (with [pool (make-hy-process-pool :max-workers max-workers)]
43 (setv futures
44 (lfor r ranges
45 (.run-in-executor loop pool count-primes-in-range r)))
46 (list (await (asyncio.gather #* futures)))))
Passing :initializer builtins.__import__ :initargs #("hy") ensures every worker process imports hy at startup before unpickling any task function from cpu_offload.hy.
A Note on Free-Threaded Python (PEP 703)
Python 3.13 introduced an experimental free-threaded build (python3.13t / python3.14t) that disables the Global Interpreter Lock (GIL), and Hy 1.3.1 runs on Python 3.9 through 3.15. You can check at runtime whether the current interpreter has the GIL enabled with python-gil-status in cpu_offload.hy:
1 (defn python-gil-status []
2 "Return a dict describing the current Python version and whether the GIL is enabled."
3 (setv check-fn (getattr sys "_is_gil_enabled" None)
4 gil-enabled (if (callable check-fn) (check-fn) True))
5 {"python_version" (get (.split sys.version) 0)
6 "gil_enabled" gil-enabled
7 "free_threaded" (not gil-enabled)})
Even on a free-threaded build, asyncio remains the right choice for high-concurrency network I/O because coroutines cost a few kilobytes of memory compared to OS threads and avoid shared-state data races by switching only at explicit await points.
6. Case Study: Porting the Web Scraper to Async (async_scraper.hy)
In the “Responsible Web Scraping” chapter (source_code_for_examples/webscraping), get_web_page.hy, get_page_data.hy, and npr_front_page_summary.hy used synchronous urllib.request.urlopen calls to download pages one by one and extract BeautifulSoup tag dictionaries and long headlines (anchor text longer than 40 characters).
In async_scraper.hy, we keep the same data extraction functions (get-element-data and parse-page-elements) and implement both a sequential scraper and a semaphore-bounded concurrent scraper:
1 (defn get-element-data [element]
2 "Extract text, tag name, class, and href from a BeautifulSoup tag.
3 Direct port of get-element-data from source_code_for_examples/webscraping/get_page_data.hy."
4 {"text" (.strip (.get-text element))
5 "name" element.name
6 "class" (.get element "class")
7 "href" (.get element "href")})
8
9
10 (defn parse-page-elements [html]
11 "Parse HTML into structured title, a, h1, and h2 element dictionaries,
12 plus a headline summary (links with text > 40 chars, matching npr_front_page_summary.hy)."
13 (setv soup (BeautifulSoup html "html.parser")
14 titles (lfor e (.find-all soup "title") (get-element-data e))
15 anchors (lfor e (.find-all soup "a") (get-element-data e))
16 h1s (lfor e (.find-all soup "h1") (get-element-data e))
17 h2s (lfor e (.find-all soup "h2") (get-element-data e))
18 long-headlines (lfor a anchors
19 :if (> (len (get a "text")) 40)
20 (get a "text")))
21 {"title" (if titles (get (get titles 0) "text") "")
22 "a" anchors
23 "h1" h1s
24 "h2" h2s
25 "summary" (.join "\n\n" long-headlines)})
26
27
28 (defn :async scrape-pages-sequential [client urls]
29 "Fetch and parse each URL one at a time (sequential baseline).
30 Return #(pages-list elapsed-seconds)."
31 (setv start (time.perf-counter)
32 results [])
33 (for [url urls]
34 (setv resp (await (fetch-with-retry client url))
35 parsed (parse-page-elements (get resp "text")))
36 (.append results {"url" url
37 "status" (get resp "status")
38 "attempts" (get resp "attempts")
39 #** parsed}))
40 #(results (- (time.perf-counter) start)))
41
42
43 (defn :async scrape-pages-concurrent [client urls [max-concurrency 4]]
44 "Fetch and parse all URLs concurrently using TaskGroup and a Semaphore.
45 Return #(pages-list elapsed-seconds)."
46 (setv start (time.perf-counter)
47 sem (asyncio.Semaphore max-concurrency)
48 tasks [])
49
50 (defn :async _scrape-one [url]
51 (setv resp (await (fetch-with-retry client url :semaphore sem))
52 parsed (parse-page-elements (get resp "text")))
53 {"url" url
54 "status" (get resp "status")
55 "attempts" (get resp "attempts")
56 #** parsed})
57
58 (with [:async tg (asyncio.TaskGroup)]
59 (for [url urls]
60 (.append tasks (.create-task tg (_scrape-one url)))))
61 (setv results (lfor t tasks (.result t)))
62 #(results (- (time.perf-counter) start)))
When we run compare-sequential-vs-concurrent over our 8 sample news pages with a 40 ms per-request simulated latency and one transient HTTP 503 failure on /news/open-models:
| Mode | In-Flight Cap | Total Requests (with 1 Retry) | Measured Time | Speedup |
|---|---|---|---|---|
scrape-pages-sequential |
1 | 9 | 396.1 ms | 1.00x |
scrape-pages-concurrent |
4 (Semaphore(4)) |
9 | 128.8 ms | 3.08x |
Both functions return the exact same list of parsed page dictionaries in the same URL order, because scrape-pages-concurrent collects (.result t) from the tasks list in the order the tasks were created.
Tests
Testing async code against live websites or live LLM servers leads to slow, flaky tests. In test_async.hy, we configure pytest-asyncio with asyncio_mode = "auto" in pyproject.toml so every (defn :async test-... []) function runs automatically inside an event loop, and we replace network sockets with httpx.MockTransport.
Here is how make-mock-transport in bounded_fetch.hy simulates latency, tracks maximum in-flight concurrency, and injects a configurable number of initial HTTP 503 failures per path:
1 (defn make-mock-transport [routes [latency-seconds 0.02] [flaky-counts None]]
2 "Build an httpx.MockTransport with artificial latency and optional transient 503s.
3 routes maps a URL path (like '/page/1') to a body string or #(status body).
4 flaky-counts maps a URL path to the number of initial 503 responses before 200."
5 (setv remaining-failures (dict (or flaky-counts {}))
6 call-counts {}
7 in-flight 0
8 max-in-flight 0)
9
10 (defn :async handler [request]
11 (nonlocal in-flight max-in-flight)
12 (setv path request.url.path)
13 (setv (get call-counts path) (+ (.get call-counts path 0) 1))
14 (+= in-flight 1)
15 (when (> in-flight max-in-flight)
16 (setv max-in-flight in-flight))
17 (try
18 (when (> latency-seconds 0)
19 (await (asyncio.sleep latency-seconds)))
20 (setv fails-left (.get remaining-failures path 0))
21 (when (> fails-left 0)
22 (setv (get remaining-failures path) (- fails-left 1))
23 (return (httpx.Response 503 :text "Service Temporarily Unavailable")))
24 (when (not (in path routes))
25 (return (httpx.Response 404 :text "Not Found")))
26 (setv entry (get routes path))
27 (if (isinstance entry tuple)
28 (httpx.Response (get entry 0) :text (get entry 1))
29 (httpx.Response 200 :text entry))
30 (finally
31 (-= in-flight 1))))
32
33 (setv transport (httpx.MockTransport handler))
34 {"transport" transport
35 "call_counts" call-counts
36 "stats" (fn [] {"max_in_flight" max-in-flight
37 "call_counts" (dict call-counts)})})
Notice the (nonlocal in-flight max-in-flight) declaration inside handler. Each time a request enters handler, it increments in-flight, updates max-in-flight, sleeps for latency-seconds, and decrements in-flight inside (finally ...). Our test in test_async.hy then asserts both that retries succeeded and that the semaphore never allowed more than 3 concurrent requests:
1 (defn :async test-bounded-fetch-semaphore-and-retries []
2 (setv routes (dfor i (range 1 9) f"/p/{i}" f"body-{i}")
3 mock (make-mock-transport routes
4 :latency-seconds 0.015
5 :flaky-counts {"/p/2" 2 "/p/6" 1})
6 urls (lfor i (range 1 9) f"https://test.local/p/{i}"))
7 (with [:async client (httpx.AsyncClient :transport (get mock "transport"))]
8 (setv res (await (fetch-all-bounded client urls
9 :max-concurrency 3
10 :max-retries 3
11 :base-delay 0.005))))
12 (assert (= (len res) 8))
13 (setv stats ((get mock "stats")))
14 (assert (<= (get stats "max_in_flight") 3))
15 (assert (= (get (get stats "call_counts") "/p/2") 3))
16 (assert (= (get (get stats "call_counts") "/p/6") 2)))
Run the full test suite with pytest:
1 $ uv run pytest -q
2 ........... [100%]
3 11 passed in 0.80s
Wrap Up
Hy gives you full access to Python’s modern concurrency stack with clean, consistent syntax: :async modifiers on functions, loops, context managers, and comprehensions; asyncio.TaskGroup and except* for structured concurrency; httpx.AsyncClient for bounded HTTP and LLM streaming; asyncio.Queue for backpressure-aware pipelines; and to-thread / ProcessPoolExecutor when work must leave the event loop.
Optional Practice Problems
Add Jitter and
Retry-AfterSupport tofetch-with-retry(Small): Modifyfetch-with-retryinbounded_fetch.hyso that when a response has status429and includes a"Retry-After"header (in seconds), it sleeps for(float (get response.headers "Retry-After"))instead of the exponential backoff formula. When"Retry-After"is absent, add uniform random jitter between0and0.005seconds to the backoff delay so concurrent retries do not fire in lockstep. Add a unit test intest_async.hyusinghttpx.MockTransport.Add an Async Summarization Stage to the Queue Pipeline (Open-Ended): Extend
run-crawl-extract-indexinpipeline.hywith a fourth stage (summarize-worker) that reads extracted documents fromdoc-queue, callscollect-chat-streamfromstreaming_llm.hy(usingmake-fake-ollama-transportby default or live Ollama when a--liveflag is set) to generate a one-sentence summary of each page, and stores the summary alongside the document in the index. Bound the summarization stage with a semaphore so at most two LLM streams run at the same time.
Further Reading
- Matthew Fowler, Python Concurrency with asyncio (Manning)
- Luciano Ramalho, Fluent Python, 2nd Edition, Chapters 19-21 (O’Reilly)