Network Programming
Distributed computing is pervasive: you need to look no further than the World Wide Web, Internet chat, etc. Of course, as a Lisp programmer, you will want to do at least some of your network programming in Lisp! Previous editions of this book provided low level socket examples. For this new edition, I removed those and instead encourage you to work at a higher level of abstraction.
Note: there is no ./src example directory for this chapter. Drakma and Hunchentoot will be used later in this book. Here we just show a few interactive examples.
We will use Edi Weitz’s Drakma HTTP client library for fetching web data and his Hunchentoot web server for building REST services that return JSON. JSON is easier to read than XML and works well with Javascript and data stores like Postgres, MongoDB, and CouchDB.
An introduction to Drakma
Edi Weitz’s Drakma library supports HTTP GET, PUT, authentication, and cookies. The top level API is drakma:http-request, which returns multiple values. The first three values are the response body, HTTP status code, and headers:
1 * (ql:quickload :drakma)
2 * (multiple-value-setq (data http-response-code headers)
3 (drakma:http-request "http://example.com"))
4
5 * data
6 "<!doctype html>..."
7 * http-response-code
8 200
9 * headers
10 ((:SERVER . "ECS")
11 (:CONTENT-TYPE . "text/html; charset=UTF-8"))
We will use Drakma later in this book. Next we will write a web app using Hunchentoot and test it with a Drakma client.
An introduction to Hunchentoot
Edi Weitz’s Hunchentoot project is a flexible library for writing web applications and services. We will also use his CL-WHO library for generating HTML from Lisp code.
1 (ql:quickload :hunchentoot)
2 (ql:quickload :cl-who)
3
4 (in-package :cl-user)
5 (defpackage hdemo
6 (:use :cl :cl-who :hunchentoot))
7 (in-package :hdemo)
8
9 (defvar *h* (make-instance 'easy-acceptor :port 3000))
10
11 (define-easy-handler (my-greetings :uri "/hello") (name)
12 (setf (hunchentoot:content-type*) "text/html")
13 (with-html-output-to-string (*standard-output* nil :prologue t)
14 (:html
15 (:head (:title "hunchentoot test"))
16 (:body
17 (:h1 "hunchentoot form demo")
18 (:form :method :post
19 (:input :type :text :name "name" :value name)
20 (:input :type :submit :value "Submit your name"))
21 (:p "Hello " (str name))))))
22
23 (hunchentoot:start *h*)
define-easy-handler creates an HTTP handler and adds it to the acceptor. The :uri keyword routes requests to this handler. The name parameter captures query or form data. CL-WHO generates HTML from Lisp code.
Fetch the page with Drakma to test:
1 * (drakma:http-request "http://127.0.0.1:3000/hello?name=Mark")
2
3 "Hello Mark"
4 200
5 ((:CONTENT-LENGTH . "10")
6 (:DATE . "Fri, 05 Jul 2013 15:57:22 GMT")
7 (:SERVER . "Hunchentoot 1.2.18")
8 (:CONNECTION . "Close")
9 (:CONTENT-TYPE . "text/plain; charset=utf-8"))
10 #<PURI:URI http://127.0.0.1:3000/hello?name=Mark>
11 #<FLEXI-STREAMS:FLEXI-IO-STREAM {10095654A3}>
12 T
13 "OK"
Complete REST Client Server Example Using JSON for Data Serialization
A reasonable way to build distributed systems is to write REST web services that serve JSON to clients. Clients might be rich web apps, other services, or smartphone applications.
We use cl-json to encode Lisp data as JSON:
1 * (ql:quickload :cl-json)
2 * (defvar y (list (list '(cat . "the cat ran") '(dog . 101)) 1 2 3 4 5))
3 * (json:encode-json-to-string y)
4 "[{\"cat\":\"the cat ran\",\"dog\":101},1,2,3,4,5]"
The following is the file src/web-hunchentoot-json.lisp:
1 (ql:quickload :hunchentoot)
2 (ql:quickload :cl-json)
3
4 (defvar *h* (make-instance 'hunchentoot:easy-acceptor :port 3000))
5
6 (hunchentoot:define-easy-handler (animal :uri "/animal") (name)
7 (setf (hunchentoot:content-type*) "text/plain")
8 (cond
9 ((string-equal name "cat")
10 (json:encode-json-to-string
11 (list (list '(average_weight . 10) '(friendly . nil))
12 "A cat can live indoors or outdoors.")))
13 ((string-equal name "dog")
14 (json:encode-json-to-string
15 (list (list '(average_weight . 40) '(friendly . t))
16 "A dog is a loyal creature, much valued by humans.")))
17 (t
18 (json:encode-json-to-string
19 (list () "unknown type of animal")))))
20
21 (hunchentoot:start *h*)
This handler returns JSON instead of HTML. Test it with Drakma:
1 * (drakma:http-request "http://127.0.0.1:3000/animal?name=dog")
2
3 "[{\"average_weight\":40,\"friendly\":true},
4 \"A dog is a loyal creature, much valued by humans.\"]"
5 200
6 * (drakma:http-request "http://127.0.0.1:3000/animal?name=cat")
7
8 "[{\"average_weight\":10,\"friendly\":null},
9 \"A cat can live indoors or outdoors.\"]"
10 200
Decode JSON back to Lisp data:
1 * (cl-json:decode-json-from-string
2 (drakma:http-request "http://127.0.0.1:3000/animal?name=dog"))
3
4 (((:AVERAGE--WEIGHT . 40) (:FRIENDLY . T))
5 "A dog is a loyal creature, much valued by humans.")
For state changes, use POST handlers:
1 (defparameter *animal-hash* (make-hash-table))
2
3 (hunchentoot:define-easy-handler (some-handler :uri "/add") (json-data)
4 (setf (hunchentoot:content-type*) "text/plain")
5 (let* ((data (cl-json:decode-json-from-string json-data))
6 (animal-name (gethash "name" data)))
7 (setf (gethash animal-name *animal-hash*) data))
8 "OK")
Network Programming Wrap Up
You have learned the basics for writing web services and clients. Later, we will wrap retrained deep learning models in Python web services and access them from Common Lisp.