Using Common Lisp with Wolfram/One
If you use Wolfram/One then the material in this short chapter may interest you. The interface that I wrote is simple: I use uiop:run-program to spawn a new process to run the Wolfram Language command line tool that writes results to a temporary file. I then use uiop:read-file-string to read the results and parse them into a convenient form for use.
Before we build and use an interface to Wolfram/One, let’s look at two screen shots of the Wolfram/One interface with examples that we will run later in Common Lisp. The first example finds entities in text:
The second example uses a deep learning model to answer a question given text containing the answer to the question:
Here is the package.lisp file for this example:
1 (defpackage #:wolfram
2 (:use #:cl #:uiop)
3 (:export #:wolfram #:cleanup-lists
4 #:find-answer-in-text #:entities))
And the wolfram.asd file:
1 (asdf:defsystem #:wolfram
2 :description "Wolfram Language interface experiments"
3 :author "Mark Watson <markw@markwatson.com>"
4 :license "Apache 2"
5 :depends-on (#:uiop #:cl-json #:myutils)
6 :components ((:file "package")
7 (:file "wolfram")))
The implementation in Wolfram.lisp is simple enough. In lines 6-8 I create a Common Lisp path object in /tmp (and absolute pathname is required) and then use file-namestring to get just the file name as a string. In lines 8-10 we are creating an operating system shell and running the Wolfram Language command line tool with arguments to execute the query and write the results to the temporary file. In lines 11-15 we read the contents of the temporary file, delete the file, and decode the returned string as JSON data.
The Data returned form calling the Wolfram Language command line tool contains excess structure that we don’t need (a sample of the raw returned data is shown later) so the function cleanup-lists shown in lines 17-19 discards heads of lists when the first value in a list or sublist is Rule or List. The function recursive-remove seen in lines 20-24 will remove all occurrences of an item from a nested list.
1 (in-package #:wolfram)
2
3 ;; General query utilities
4
5 (defun wolfram (statement)
6 (let ((temp-file-path
7 (file-namestring (uiop:tmpize-pathname "/tmp/wolfram"))))
8 (uiop:run-program (concatenate 'string "wolframscript -code 'Export[\""
9 temp-file-path "\"," statement
10 ",\"ExpressionJSON\"]'"))
11 (let* ((ret (uiop:read-file-string temp-file-path)))
12 (delete-file temp-file-path)
13 (with-input-from-string (s (myutils:replace-all
14 (myutils:replace-all ret "\"'" "\"") "'\"" "\""))
15 (json:decode-json s)))))
16
17 (defun cleanup-lists (r)
18 (cdr (recursive-remove "Rule" (recursive-remove "List" r))))
19
20 (defun recursive-remove (item tree)
21 (if (atom tree)
22 tree
23 (mapcar (lambda (nested-list) (recursive-remove item nested-list))
24 (remove item tree :test #'equal))))
25
26 ;; Higher level utilities for specific types of queries
27
28 (defun entities (text)
29 (let* ((noquotes (myutils:replace-all (myutils:replace-all text "\"" " ") "'" " "))
30 (query2
31 (concatenate
32 'string "TextCases['" noquotes
33 "', {'City', 'Country', 'Date', 'Person'} ->"
34 " {'String', 'Interpretation', 'Probability'}]"))
35 (query (myutils:replace-all query2 "'" "\"")))
36 (remove-if #'(lambda (a) (null (cadr a)))
37 (cleanup-lists (wolfram query)))))
38
39 (defun find-answer-in-text (text question)
40 (let* ((nqtext (myutils:replace-all (myutils:replace-all text "\"" " ") "'" " "))
41 (nqquestion (myutils:replace-all
42 (myutils:replace-all question "\"" " ") "'" " "))
43 (query2 (concatenate 'string "FindTextualAnswer['" nqtext
44 "', '" nqquestion "']"))
45 (query (myutils:replace-all query2 "'" "\"")))
46 (wolfram query)))
The last two functions in the last code listing, entities and find-answer-in-text are higher level functions intended to work with the Wolfram Language procedures TextCases (see Wolfram documentation for TextCases) and FindTextualAnswer (see Wolfram documentation for FindTextualAnswer).
The functions cleanup-lists and recursive-remove can be used to clean up results. First, we will just call function wolfram and show the raw results:
1 $ sbcl
2 * (ql:quickload "wolfram")
3 To load "wolfram":
4 Load 1 ASDF system:
5 wolfram
6 ; Loading "wolfram"
7 [package myutils].................................
8 [package wolfram]
9 ("wolfram")
10 * (setf example "TextCases['NYC, Los Angeles, and Chicago are the largest cities in the USA in 2018 according to Pete Wilson.', {'City', 'Country', 'Date', 'Person'} -> {'String', 'Interpretation', 'Probability'}]")
11 "TextCases['NYC, Los Angeles, and Chicago are the largest cities in the USA in 2018 according to Pete Wilson.', {'City', 'Country', 'Date', 'Person'} -> {'String', 'Interpretation', 'Probability'}]"
12 * (setf example-str (myutils:replace-all example "'" "\""))
13 "TextCases[\"NYC, Los Angeles, and Chicago are the largest cities in the USA in 2018 according to Pete Wilson.\", {\"City\", \"Country\", \"Date\", \"Person\"} -> {\"String\", \"Interpretation\", \"Probability\"}]"
14 * (setf results (wolfram:wolfram example-str))
15 * (pprint results)
16
17 ("Association"
18 ("Rule" "City"
19 ("List"
20 ("List" "NYC" ("Entity" "City" ("List" "NewYork" "NewYork" "UnitedStates"))
21 0.75583166)
22 ("List" "Los Angeles"
23 ("Entity" "City" ("List" "LosAngeles" "California" "UnitedStates"))
24 0.84206486)
25 ("List" "Chicago"
26 ("Entity" "City" ("List" "Chicago" "Illinois" "UnitedStates"))
27 0.91092855)))
28 ("Rule" "Country"
29 ("List" ("List" "USA" ("Entity" "Country" "UnitedStates") 0.9285077)))
30 ("Rule" "Date"
31 ("List"
32 ("List" "2018" ("DateObject" ("List" 2018) "Year" "Gregorian" -7.0)
33 0.8364356)))
34 ("Rule" "Person"
35 ("List"
36 ("List" "Pete Wilson" ("Entity" "Person" "PeteWilson::s7259") 0.9274548))))
37 *
Now we clean up the output:
1 * (defvar results-cleaned (wolfram:cleanup-lists results))
2 * (pprint results-cleaned)
3
4 (("City"
5 (("NYC" ("Entity" "City" ("NewYork" "NewYork" "UnitedStates")) 0.75583166)
6 ("Los Angeles" ("Entity" "City" ("LosAngeles" "California" "UnitedStates"))
7 0.84206486)
8 ("Chicago" ("Entity" "City" ("Chicago" "Illinois" "UnitedStates"))
9 0.91092855)))
10 ("Country" (("USA" ("Entity" "Country" "UnitedStates") 0.9285077)))
11 ("Date" (("2018" ("DateObject" (2018) "Year" "Gregorian" -7.0) 0.8364356)))
12 ("Person" (("Pete Wilson" ("Entity" "Person" "PeteWilson::s7259") 0.9274548))))
13 *
Next we will try the two higher-level utility functions. The first example shows finding entities in text:
1 CL-USER 21 > (pprint
2 (wolfram:entities "Sedona Arizona is home to Mark Louis Watson"))
3
4 (("City"
5 (("Sedona" ("Entity" "City" ("Sedona" "Arizona" "UnitedStates")) 0.8392784)))
6 ("Person" (("Mark Louis Watson" "Mark Louis Watson" 0.9023427))))
The second example uses a Wolfram pre-trained deep learning model for question answering:
1 CL-USER 22 > (pprint
2 (wolfram::find-answer-in-text "International Business Machines Corporation (IBM) is an American multinational technology company headquartered in Armonk, New York, with operations in over 170 countries. The company began in 1911, founded in Endicott, New York, as the Computing-Tabulating-Recording Company (CTR) and was renamed \"International Business Machines\" in 1924. IBM is incorporated in New York."
3 "where is IBM is headquartered?"))
4
5 "Armonk, New York"
If you use Wolfram/One then these examples should get you started wrapping other Wolfram Language functionality for use in your Common Lisp applications.