Using the Brave Search APIs
Note: I started using the Brave search APIs in June 2024 and replaced the Microsoft Bing search chapter in previous editions with the following material.
You will need to get a free API key at https://brave.com/search/api/ to use the following code examples. You can use the search API 2000 times a month for free or pay $5/month to get 20 million API calls a month.
Setting an Environment Variable for the Access Key for Brave Search APIs
Once you get a key for https://brave.com/search/api/ set the following environment variable:
1 export BRAVE_SEARCH_API_KEY=BSGhQ-Nd-......
That is not my real subscription key!
Example Search Script
Get the code for this example using (change directory to your Quicklist local-projects directory):
1 cd ~/.roswell/local-projects/brave_search # if you use Roswell in install and update SBCL, etc.
2 cd ~/quicklisp/local-projects # if you DO NOT use Roswell
3 git clone https://github.com/mark-watson/brave_search.git
4 cd brave_search
It takes very little Common Lisp code to access the Brave search APIs. The function websearch makes a generic web search query. I will list the entire library with comments to follow:
1 ;; Copyright Mark Watson 2024. All Rights Reserved. https://markwatson.com
2 ;; Apache 2 license.
3
4 (in-package #:brave_search)
5
6 (defun websearch (query)
7 (let* ((key (uiop:getenv "BRAVE_SEARCH_API_KEY"))
8 (command
9 (concatenate
10 'string
11 "curl https://api.search.brave.com/res/v1/web/search?q="
12 (drakma:url-encode query :utf-8)
13 " -H \"X-Subscription-Token: " key "\""
14 " -H \"Content-Type: application/json\""))
15 (response
16 (uiop:run-program command :output :string)))
17 ;;(print response) ;; weird: comment this out, and a runtime error is thrown
18 (with-input-from-string
19 (s response)
20 (let ((results (cdar (cddr (assoc :web (json:decode-json s))))))
21 (mapcar (lambda (x)
22 (let ((title (cdr (assoc :title x)))
23 (url (cdr (assoc :url x)))
24 (description (cdr (assoc :description x))))
25 (list url title description)))
26 results)))))
27
28 ;; Example usage:
29 ;; (brave_search:websearch "Sedona Arizona")
We get the Brave access key and the search API endpoint in lines 8-9. Lines 10-16 create a complete call to the *curl command line utility. We spawn a process to run curl and capture the string output in the variable response in lines 17-18. You might want to add a few print statements to see typical values for the variables command and response. The response data is JSON data encoded in a string, with straightforward code in lines 19-28 to parse out the values we want.
The following repl listing shows this library in use (most output not shown):
1 $ sbcl
2 * (ql:quickload "brave_search")
3 To load "brave_search":
4 Load 1 ASDF system:
5 brave_search
6 ; Loading "brave_search"
7 ..................
8 ("brave_search")
9 * (brave_search:websearch "Sedona Arizona")
10 (("https://visitsedona.com/"
11 "Visit Sedona | The official site of the Sedona Tourism Bureau"
12 "The official site of the <strong>Sedona</strong>, AZ tourism bureau. Find out the best places to stay, eat, and relax in our beautiful central <strong>Arizona</strong> resort town.")
13 ("https://www.sedonaaz.gov/" "City of Sedona | Home"
14 "The City of <strong>Sedona</strong> wastewater department has experienced ...