Hacking the SBCL REPL
Note to readers: This chapter is a work in progress. Additional topics will be added in future updates.
The SBCL REPL is already a powerful environment for interactive development, but Common Lisp’s extensibility lets us reshape it into something more. In this chapter we explore how to customize the SBCL REPL by modifying the ~/.sbclrc startup file, adding shell command execution, custom reader macros, and quality-of-life improvements that blur the line between a Lisp REPL and a Unix shell. The two customizations we implement are:
- Using
#!to execute shell commands. - Adding an AI coding agent that is always available in the REPL and instantiated with
#?.
Shell Access via a Reader Macro
To achieve seamless, shell-like command execution in the SBCL REPL without requiring s-expression wrappers, the most idiomatic and robust mechanism is Common Lisp’s reader macros.
Instead of hacking the REPL’s top-level evaluation loop directly, we modify the readtable so that the reader itself intercepts special characters. Here is how the pieces fit together:
- Interception — By binding
#!as a dispatch macro character, the Lisp reader hands control to a custom function the moment it encounters the#!sequence. - Consumption — The macro function consumes the rest of the input line as a raw string, completely bypassing the standard Lisp parser for that line.
- Evaluation — The macro returns a valid Lisp form (a function call) containing the parsed string. The REPL naturally evaluates this form.
- Process Management — Shell commands like
lsorgrepare passed directly to the underlying OS shell usinguiop:run-program. - State Mutation — Process-level state changes, specifically
cd, cannot be delegated to a subshell because the subshell’s environment terminates immediately. The macro must trapcd, execute it internally usinguiop:chdir, and update Lisp’s*default-pathname-defaults*to keep the Lisp environment in sync with the OS-level working directory.
Why
#!instead of bare!? The exclamation mark is a valid constituent character in Common Lisp symbols. Libraries likecl-jsonuse symbols such asrun!internally. A bare!reader macro hijacks this character globally — when SBCL loads any.asdfile or source file containing!in a symbol name, the reader macro fires, consumes the rest of the line as a shell command, and leaves the reader with unbalanced parentheses. Using#!(a dispatch macro on#) avoids this entirely because#is already reserved as a dispatching macro character and never appears as part of a symbol name.
Adding the Code to ~/.sbclrc
Add the following code to the end of your ~/.sbclrc file. It requires ASDF (which ships with SBCL) for access to UIOP’s process and filesystem utilities:
1 (require :asdf)
2
3 (defun run-shell-command (line)
4 "Executes a raw shell command string and forcefully prints captured output."
5 (if (string= line "")
6 (values)
7 (let* ((space-pos (position #\Space line))
8 (cmd (if space-pos (subseq line 0 space-pos) line))
9 (args (if space-pos
10 (string-trim " " (subseq line space-pos))
11 "")))
12 (cond
13 ;; Handle cd internally
14 ((string-equal cmd "cd")
15 (let ((target-dir (if (string= args "")
16 (namestring (user-homedir-pathname))
17 args)))
18 (uiop:chdir target-dir)
19 (setf *default-pathname-defaults* (uiop:getcwd))
20 (format t "~A~%" (uiop:getcwd))
21 (force-output)
22 (values)))
23
24 ;; Pass to system shell, capture output as strings
25 (t
26 (multiple-value-bind (stdout stderr exit-code)
27 (uiop:run-program line
28 :output :string
29 :error-output :string
30 :ignore-error-status t)
31 (declare (ignore exit-code))
32 ;; Print captured strings explicitly to Lisp's stdout
33 (when (plusp (length stdout))
34 (princ stdout))
35 (when (plusp (length stderr))
36 (princ stderr))
37 (force-output)
38 (values)))))))
39
40 (defun bang-reader (stream disp-char sub-char)
41 "Dispatch reader macro for #! — executes a shell command."
42 (declare (ignore disp-char sub-char))
43 (let ((line (with-output-to-string (out)
44 ;; Peek at the next character. If it's not a newline
45 ;; or EOF, read it.
46 (loop for c = (peek-char nil stream nil nil)
47 while (and c
48 (char/= c #\Newline)
49 (char/= c #\Return))
50 do (write-char (read-char stream) out)))))
51 `(run-shell-command ,(string-trim " " line))))
52
53 ;; Bind #! as a dispatch macro character
54 (set-dispatch-macro-character #\# #\! #'bang-reader)
How It Works
The run-shell-command Function
The run-shell-command function is the workhorse. It takes a single string argument — the raw text after the #! sequence — and dispatches it:
- Empty input — returns immediately with no values.
cdcommand — handled internally. Acdexecuted in a subshell would change that subshell’s directory, then immediately exit — leaving the parent Lisp process unchanged. Instead, we calluiop:chdirto change the OS-level working directory and update*default-pathname-defaults*so that subsequent Lisp file operations (like(load "foo.lisp")) resolve relative to the new directory.- Everything else — delegated to the OS shell via
uiop:run-program. The:output :stringand:error-output :stringkeyword arguments capture both stdout and stderr as strings. We then print them explicitly withprinc(which omits quotation marks, unlikeprintorformat ~S) and callforce-outputto flush the stream immediately.
The :ignore-error-status t argument prevents uiop:run-program from signaling a condition on non-zero exit codes — important for commands like grep that return exit code 1 when no matches are found.
The bang-reader Dispatch Macro Function
The bang-reader function is installed as a dispatch reader macro for the #! character sequence. When the Lisp reader encounters #!, it calls this function with three arguments: the input stream, the dispatch character (#), and the sub-character (!). We declare both character arguments as ignored since we only need the stream.
Rather than using read-line (which would consume the newline and potentially confuse the REPL’s line tracking), the function uses peek-char in a loop to read characters one at a time until it hits a newline or end-of-file. This leaves the newline in the stream for the REPL to consume normally.
The function returns a backquoted form:
1 `(run-shell-command ,(string-trim " " line))
This is a valid Lisp form that the REPL’s evaluator processes normally — calling run-shell-command with the captured string as its argument.
The set-dispatch-macro-character Binding
The final line installs the dispatch macro:
1 (set-dispatch-macro-character #\# #\! #'bang-reader)
After this executes (at SBCL startup, via ~/.sbclrc), any #! sequence at the REPL is intercepted before the standard Lisp reader ever sees the rest of the line. Unlike a bare ! reader macro, this approach is safe — it cannot interfere with symbol names containing exclamation marks, because # is already reserved as a non-constituent dispatching prefix.
Example Session
After adding this code to ~/.sbclrc and restarting SBCL:
1 $ sbcl
2 * #! ls -la *.lisp
3 -rw-r--r-- 1 mark staff 1234 May 11 16:30 example.lisp
4 -rw-r--r-- 1 mark staff 567 May 11 15:00 utils.lisp
5
6 * #! pwd
7 /Users/mark/projects
8
9 * #! cd /tmp
10
11 /private/tmp/
12
13 * #! pwd
14 /private/tmp/
15
16 * #! cd
17
18 /Users/mark/
19
20 * (+ 1 2)
21 3
Notice that normal Lisp expressions still work exactly as before. The #! macro only activates when #! appears in the input. You can freely alternate between shell commands and Lisp expressions within the same REPL session.
AI Coding Agent Integration
The #! reader macro from the previous section gave us shell access from the REPL. In this section we go further by integrating an AI coding agent directly into the REPL so we can ask questions, diagnose errors, and generate code without leaving SBCL.
The cl-ai-coding-agent package (developed in the “Building an AI Coding Assistant for Common Lisp”) provides a function coding-agent-query that takes a string prompt and returns the agent’s response. The agent can autonomously list directories, read files, write new files, and diagnose stacktraces using Gemini’s function-calling API. But calling it directly requires quoting strings:
1 (cl-ai-coding-agent:coding-agent-query
2 "Write a function that sorts a list of strings")
This syntactic friction slows down the interactive workflow. We want three levels of integration:
- An
aimacro — eliminates string quoting by stringifying unevaluated symbols, so we can type natural language as Lisp forms. - A
#?reader macro — captures an entire line as a prompt, matching the#!pattern from the shell integration.
The Code
Add the following code to your ~/.sbclrc file, after the shell integration code from the previous section. It assumes cl-ai-coding-agent is installed in your Quicklisp local-projects directory:
1 ;;; ---- AI Coding Agent REPL Integration ----
2
3 ;; Load the agent on startup
4 (ql:quickload :cl-ai-coding-agent :silent t)
5
6 ;; 1. The AI macro -- type natural language without quotes
7 (defmacro ai (&rest words)
8 "Ask the AI coding agent a question using natural
9 language without string quotes.
10 Usage: (ai write a function that sorts strings)"
11 (let ((prompt (format nil "~{~A~^ ~}" words)))
12 `(progn
13 (format t "~&~A~%"
14 (cl-ai-coding-agent:coding-agent-query
15 ,prompt))
16 (values))))
17
18 ;; 2. The #? reader macro -- one-line AI queries
19 (defun ai-query-reader (stream disp-char sub-char)
20 "Dispatch reader macro for #? -- sends the rest
21 of the line to the AI coding agent."
22 (declare (ignore disp-char sub-char))
23 (let ((line (with-output-to-string (out)
24 (loop for c = (peek-char nil stream
25 nil nil)
26 while (and c
27 (char/= c #\Newline)
28 (char/= c #\Return))
29 do (write-char
30 (read-char stream) out)))))
31 `(progn
32 (format t "~&~A~%"
33 (cl-ai-coding-agent:coding-agent-query
34 ,(string-trim " " line)))
35 (values))))
36
37 (set-dispatch-macro-character #\# #\? #'ai-query-reader)
38
39 ;; 3. Automatic error interception
40 (defun ai-diagnose-error (condition)
41 "Format a condition into a diagnostic prompt and
42 send it to the AI coding agent."
43 (let* ((text (format nil "~A" condition))
44 (response
45 (cl-ai-coding-agent:coding-agent-query
46 text)))
47 (format t "~&~%--- AI Diagnosis ---~%~A~%~
48 --- End Diagnosis ---~%"
49 response)))
How It Works
The ai Macro
The ai macro accepts unquoted natural language tokens and concatenates them into a prompt string at macro-expansion time:
1 (ai write a quicksort function)
expands to:
1 (progn
2 (format t "~&~A~%"
3 (cl-ai-coding-agent:coding-agent-query
4 "write a quicksort function"))
5 (values))
The (values) suppresses the return value to keep the REPL output clean — we only want to see the agent’s printed response, not a redundant return string. Because the macro stringifies symbols, any valid Lisp identifier characters work: letters, digits, hyphens. However, characters that the Lisp reader treats specially — parentheses, commas, quotes, semicolons — will cause read errors. For prompts that need those characters, use coding-agent-query with an explicit string, or the #? reader macro.
The #? Reader Macro
The #? dispatch macro works identically to #! but routes to the AI agent instead of the shell:
1 * #? write the common list file test.lisp that prints numbers form 1 to 10
2
3 I have created the file `test.lisp` with the following Common Lisp code:
4
5 ``lisp
6 (loop for i from 1 to 10
7 do (format t "~D~%" i))
8 ``
Like #!, it consumes the rest of the input line as a raw string, bypassing the Lisp reader entirely. This means any characters — including parentheses, quotes, and semicolons — are treated as plain text. This makes #? ideal for pasting error messages or asking questions that contain Lisp syntax.
Diagnosing Stacktraces from Files
Stacktraces often contain double-quote characters and span multiple lines, making them awkward to paste into a Lisp string literal. The coding-agent-query-file function reads the contents of a file and sends them as the prompt:
1 ;; Save a stacktrace to a file (e.g., from terminal):
2 ;; pbpaste > /tmp/error.txt
3 ;;
4 ;; Then in the REPL:
5 (cl-ai-coding-agent:coding-agent-query-file
6 "/tmp/error.txt")
7
8 "### Error Analysis: DIVISION-BY-ZERO
9
10 **1. Root Cause**
11 The error was caused by the expression `(/ 10 0)`, which attempts to perform an integer division where the divisor is zero.
12
13 **2. What Went Wrong**
14 In Common Lisp (and most programming languages), division by zero is an undefined operation. When the SBCL kernel's `INTEGER-/-INTEGER` function encountered `0` as the denominator, it signaled a `DIVISION-BY-ZERO` condition. This halted execution and invoked the debugger.
15
16 **3. Concrete Fix**
17 To prevent this error, you should validate that the divisor is not zero before performing the division. Depending on your use case, you can use a conditional check or handle the condition.
18
19 #### Option A: Conditional Check (Recommended)
20 Before dividing, check if the denominator is zero using `zerop`.
21
22 ``lisp
23 (let ((numerator 10)
24 (denominator 0))
25 (if (zerop denominator)
26 (format t \"Error: Cannot divide by zero.\")
27 (/ numerator denominator)))
28 ``
29
30 #### Option B: Using `handler-case`
31 If the division is part of a larger computation and you want to catch the error gracefully:
32
33 ``lisp
34 (handler-case (/ 10 0)
35 (division-by-zero ()
36 (format t \"Caught division by zero! Returning NIL.\")
37 nil))
38 ``
39
40 #### Option C: Using `ignore-errors`
41 If you simply want the expression to return `NIL` instead of crashing:
42
43 ``lisp
44 (ignore-errors (/ 10 0)) ; Returns NIL and the condition object as a second value
45 ``"
Example Session
After adding the integration code to ~/.sbclrc and restarting SBCL:
1 $ sbcl
2 * (ai what is a hash table in common lisp)
3 A hash table in Common Lisp is a data structure that maps
4 keys to values using a hash function for fast lookups.
5 You create one with MAKE-HASH-TABLE and access entries
6 with GETHASH:
7
8 (defvar *ht* (make-hash-table))
9 (setf (gethash :name *ht*) "Alice")
10 (gethash :name *ht*) ; => "Alice"
11
12 * #? list the .lisp files in this directory and describe them
13 The current directory contains:
14 - agent.lisp: Core agent logic with the tool-use loop
15 - package.lisp: Package definition and exports
16 - tools.lisp: File-system tool implementations
17 - cl-ai-coding-agent.asd: ASDF system definition
18
19 * #! ls *.lisp
20 agent.lisp package.lisp tools.lisp
21
22 * (+ 1 2)
23 3
24
25 * #? write a file test.lisp to run a simple test of groq.lisp
26 I have created a file named `test.lisp` in the current directory to test the `groq` library. This script loads the system using ASDF and performs a simple completion request.
27
28 The contents of `test.lisp` are as follows:
29
30 ``lisp
31 ;;;; test.lisp
32 ;;;; Simple test for the groq library
33
34 (require :asdf)
35 (asdf:load-asd (truename "groq.asd"))
36 (asdf:load-system :groq)
37
38 (defun run-groq-test ()
39 (format t "Starting Groq API test...~%")
40 (unless (uiop:getenv "GROQ_API_KEY")
41 (format t "Warning: GROQ_API_KEY environment variable is not set.~%")
42 (return-from run-groq-test))
43
44 (let* ((prompt "Say 'Hello, World!' in Common Lisp.")
45 (response (groq:groq-completion prompt))
46 (content (groq:groq-extract-content response)))
47 (format t "Prompt: ~A~%" prompt)
48 (format t "Extracted Content:~%~A~%" content)))
49
50 (run-groq-test)
51 ``
52
53 To run this test, ensure you have your `GROQ_API_KEY` environment variable set, then execute:
54 ``bash
55 sbcl --load test.lisp --quit
56 ``
57 *
Notice how all integration levels coexist. The ai macro handles simple natural-language queries, #? handles anything with special characters, #! still works for shell commands, and Standard Lisp expressions continue to work normally.