Using Perplexity Sonar Model for Combined Web Search and LLM Based Reasoning

This chapter combines web search with using LLMs for reasoning. We will use a commercial API from Perplexity. You need a Perplexity API key: https://docs.perplexity.ai/home

I buy $5 of credits at a time and these credits usually last me a few months of experimenting, your mileage may vary. If you use this API in production you will want to check the pricing information and use the cheapest model that will work for you: https://docs.perplexity.ai/guides/pricing

Perplexity’s Sonar API represents a significant advancement in leveraging large language models by seamlessly integrating real-time web search capabilities directly into the reasoning process. Unlike traditional approaches that might require complex orchestration between separate search APIs and LLMs, Sonar allows the language model to autonomously access and synthesize current information from the web as it formulates a response. This dynamic interaction ensures that the generated outputs are not limited to the potentially outdated knowledge contained within the model’s training data, thereby providing more accurate, relevant, and up-to-date answers. The API offers different models, such as Sonar, Sonar Pro, Sonar Reasoning, and Sonar Deep Research, each tailored for varying levels of search depth, reasoning complexity, and the ability to handle intricate, multi-step queries. A key benefit of using the Perplexity API for this combined approach is the ability to ground LLM responses in verifiable information, often accompanied by citations to the sources found during the real-time search. This enhances the trustworthiness and reliability of the AI’s output, which is particularly valuable for applications requiring factual accuracy. You, dear reader, can utilize the Perplexity API to build applications that require access to current events, perform in-depth research, or provide answers to questions where the information is constantly changing, effectively overcoming the limitations of models that rely solely on static knowledge and mitigating the risk of generating incorrect or fabricated information.

Perplexity Sonar API architecture: combined web search and LLM reasoning

A Hy Language Client Library for Perplexity

The following Hy code defines two functions. The function search-llm-with-citations sends a user query to the Perplexity API and returns both the answer text and the list of web sources the answer cites. The convenience function search-llm returns just the answer. Perplexity offers an OpenAI compatibility feature, so we use the Python OpenAI library configured with the Perplexity API’s base URL and an API key read from the environment variable PERPLEXITY_API_KEY. If the key is not set, we raise an exception with a clear message rather than letting the request fail with a confusing authentication error. A system message instructs the model on its role as a programming and tech assistant utilizing web search and reasoning; the request is sent to the sonar model through the chat completions endpoint.

Here is a listing of the file perplexity_search_llm/search_llm.hy:

 1 (import os)
 2 (import openai)
 3 
 4 ;; Perplexity offers an OpenAI-compatible API, so we use the openai library
 5 ;; pointed at Perplexity's base URL.
 6 
 7 (setv system-message
 8       {"role" "system"
 9        "content" "You are an artificial intelligence assistant for helping a user with programming and tech questions using web search and reasoning."})
10 
11 (defn search-llm-with-citations [query [model "sonar"]]
12   "Ask a Perplexity Sonar model QUERY. Returns [answer citations], where
13   citations is the list of source URLs the answer's [1], [2], ... refer to."
14   (setv api-key (os.environ.get "PERPLEXITY_API_KEY"))
15   (when (not api-key)
16     (raise (RuntimeError "Set the PERPLEXITY_API_KEY environment variable")))
17   (setv client (openai.OpenAI :api-key api-key
18                               :base-url "https://api.perplexity.ai"))
19   (setv response (client.chat.completions.create
20                    :model model
21                    :messages [system-message {"role" "user" "content" query}]))
22   ;; citations is a Perplexity extension to the OpenAI response format
23   [(. (get response.choices 0) message content)
24    (or (getattr response "citations" None) [])])
25 
26 (defn search-llm [query]
27   "Ask a Perplexity Sonar model QUERY and return just the answer text."
28   (get (search-llm-with-citations query) 0))
29 
30 (when (= __name__ "__main__")
31   (setv [answer citations]
32         (search-llm-with-citations "Consultant Mark Watson has written many books on AI, Lisp and the semantic web. What musical instruments does he play?"))
33   (print answer)
34   (print "\nSources:")
35   (for [[i url] (enumerate citations 1)]
36     (print f"[{i}] {url}")))

Answers from Sonar contain numbered references like [1]. These refer to the URLs in the citations list, a Perplexity extension to the OpenAI response format. Because the OpenAI library does not know about this extra field, we read it with getattr and a default value, so the code still works if a response has no citations. Hy’s (. object attribute1 attribute2) form on the first element of the returned list is a compact way to write object.attribute1.attribute2.

Example Output

Here is the output from the test query at the bottom of the last program listing:

 1 $ uv run hy search_llm.hy
 2 Mark Watson plays **guitar, didgeridoo, and American Indian flute**.[1]
 3 
 4 Sources:
 5 [1] https://markwatson.com/llms.txt
 6 [2] https://www.chessprogramming.org/Mark_Watson
 7 [3] https://www.linkedin.com/in/mark-watson-23bb8417
 8 [4] https://www.linkedin.com/in/marklwatson
 9 [5] https://www.broadjam.com/MarkWatson
10 [6] https://github.com/mark-watson
11 [7] https://mastodon.social/@mark_watson
12 [8] https://markwatson.com/
13 [9] https://mark-watson.blogspot.com/
14 [10] https://me.sh/profile/mark-watson-a91522f0

Wrap Up for Using Perplexity

While the Perplexity API could be expensive to use in high API call volume production applications, I find it to be very useful for simplifying my code when I need to combine web search with LLM use.

Optional Practice Problems

To deepen your understanding of using Perplexity’s API with Hy, try implementing the following exercises by modifying the code in search_llm.hy:

  • Choosing a Model: Add an optional command line argument to search_llm.hy that selects the Sonar model (for example sonar-pro or sonar-reasoning) and pass it to search-llm-with-citations. Compare answer quality, citations, and cost for a few of your own questions.
  • Implementing Streaming Responses: For longer queries or slower network connections, waiting for the entire answer can result in a laggy user experience. Modify the API call in search_llm.hy to use streaming (by passing :stream True to client.chat.completions.create) and iterate over the chunks as they arrive to print the generated text in real-time.
  • Comparing Search Models: Perplexity provides different models such as sonar, sonar-pro, and sonar-reasoning. Modify the script to allow passing a model name as an argument, and compare the response quality, citation depth, and execution time between the default sonar model and sonar-reasoning.