Wikidata API Using SPARQL Queries
Wikidata is a free, collaborative, and multilingual knowledge base that functions as the central structured data repository for the Wikimedia ecosystem, including projects like Wikipedia, Wikivoyage, and Wiktionary. Launched in 2012, its mission is to create a common source of open data that can be used by anyone, anywhere. Unlike Wikipedia, which contains prose articles, Wikidata stores information in a machine-readable format structured around items (representing any concept or object), which are described by properties (like “population” or “author”) and corresponding values. For example, the item for “Earth” has a property “instance of” with the value “planet”. This structured approach allows for data consistency across hundreds of language editions of Wikipedia and enables powerful, complex queries through its SPARQL endpoint. By providing a centralized, queryable, and interlinked database of facts, Wikidata not only supports Wikimedia projects but also serves as a crucial resource for researchers, developers, and applications worldwide that require reliable and openly licensed structured information.
Example Code
This code in file wikidata.ss provides a client for interacting with the Wikidata Query Service, a powerful public SPARQL endpoint for accessing the vast, collaboratively edited knowledge base of Wikidata. The code encapsulates the entire process of querying this service, starting with a raw SPARQL query string. It properly formats the request by URL-encoding the query, constructs the full request URL, and sets the appropriate HTTP headers, including the Accept header for the SPARQL JSON results format and a User-Agent header, which is a requirement for responsible API usage. Upon receiving a response, the module parses the JSON data and then transforms the verbose, nested structure of the standard SPARQL results format into a more convenient and idiomatic Gerbil Scheme data structure—either a list of hash tables or a list of association lists. The file also includes several example functions that demonstrate how to query for specific facts, such as Grace Hopper’s birth date, and how to perform more complex, multi-stage queries, like first finding the unique identifiers for entities like Bill Gates and Microsoft and then discovering the relationships that connect them within the knowledge graph.
1 ;; File: wikidata.ss
2 (import :std/net/request
3 :std/text/json
4 :std/net/uri) ; For URL encoding
5 (import :std/format)
6
7 (export query-wikidata query-wikidata/alist query-dbpedia alist-ref test1 test1-ua test2 test2-ua)
8
9 ;; Helper to process the SPARQL JSON results format
10 (def (process-sparql-results json-data)
11 (let* ((results (hash-ref json-data 'results))
12 (bindings (hash-ref results 'bindings)))
13 (map (lambda (binding)
14 (let ((result-hash (make-hash-table)))
15 (hash-for-each
16 (lambda (var-name value-obj)
17 (hash-put! result-hash
18 (if (symbol? var-name) var-name (string->symbol var-name))
19 (hash-ref value-obj 'value)))
20 binding)
21 result-hash))
22 bindings)))
23
24 ;; Convenience: look up a key in an alist. Accepts symbol or string keys.
25 ;; Usage: (alist-ref 'var row [default])
26 (def (alist-ref key row . default)
27 (let* ((sym (if (symbol? key) key (string->symbol key)))
28 (p (assq sym row)))
29 (if p (cdr p) (if (pair? default) (car default) #f))))
30
31 ;; Same as above but returns an alist per row: ((var . value) ...)
32 (def (process-sparql-results-alist json-data)
33 (let* ((results (hash-ref json-data 'results))
34 (bindings (hash-ref results 'bindings)))
35 (map (lambda (binding)
36 (let ((row '()))
37 (hash-for-each
38 (lambda (var-name value-obj)
39 (let* ((sym (if (symbol? var-name) var-name (string->symbol var-name)))
40 (val (hash-ref value-obj 'value)))
41 (set! row (cons (cons sym val) row))))
42 binding)
43 (reverse row)))
44 bindings)))
45
46 ;; Query the Wikidata Query Service (WDQS)
47 ;; - Uses GET with URL-encoded query and JSON format
48 ;; - Sends a User-Agent per WDQS guidelines; callers can override
49 (def (query-wikidata sparql-query . opts)
50 (let* ((endpoint "https://query.wikidata.org/sparql")
51 (encoded-query (uri-encode sparql-query))
52 (request-url (string-append endpoint "?query=" encoded-query "&format=json"))
53 (user-agent (if (pair? opts) (car opts) "gerbil-wikidata/0.1 (+https://example.org; contact@example.org)"))
54 (headers `(("Accept" . "application/sparql-results+json")
55 ("User-Agent" . ,user-agent))))
56 (let ((response (http-get request-url headers: headers)))
57 (if (= (request-status response) 200)
58 (let ((response-json (request-json response)))
59 (process-sparql-results response-json))
60 (error "SPARQL query failed"
61 status: (request-status response)
62 body: (request-text response))))))
63
64 ;; Alist variant returning rows as association lists
65 (def (query-wikidata/alist sparql-query . opts)
66 (let* ((endpoint "https://query.wikidata.org/sparql")
67 (encoded-query (uri-encode sparql-query))
68 (request-url (string-append endpoint "?query=" encoded-query "&format=json"))
69 (user-agent (if (pair? opts) (car opts) "gerbil-wikidata/0.1 (+https://example.org; contact@example.org)"))
70 (headers `(("Accept" . "application/sparql-results+json")
71 ("User-Agent" . ,user-agent))))
72 (let ((response (http-get request-url headers: headers)))
73 (if (= (request-status response) 200)
74 (let ((response-json (request-json response)))
75 (process-sparql-results-alist response-json))
76 (error "SPARQL query failed"
77 status: (request-status response)
78 body: (request-text response))))))
79
80 ;; Backward-compatibility alias for previous DBPedia function name
81 (def (query-dbpedia . args)
82 (apply query-wikidata args))
83
84 ;; Example Usage: fetch birth date and birthplace label for Grace Hopper
85 (def (test2)
86 (let ((query
87 (string-append
88 "PREFIX wd: <http://www.wikidata.org/entity/>\n"
89 "PREFIX wdt: <http://www.wikidata.org/prop/direct/>\n"
90 "PREFIX wikibase: <http://wikiba.se/ontology#>\n"
91 "PREFIX bd: <http://www.bigdata.com/rdf#>\n"
92 "SELECT ?birthDate ?birthPlaceLabel WHERE {\n"
93 " wd:Q7249 wdt:P569 ?birthDate .\n"
94 " wd:Q7249 wdt:P19 ?birthPlace .\n"
95 " SERVICE wikibase:label { bd:serviceParam wikibase:language \"en\". }\n"
96 "}")))
97 (let ((results (query-wikidata/alist query)))
98 (for-each
99 (lambda (result)
100 (display (format "Birth Date: ~a\n" (alist-ref 'birthDate result)))
101 (display (format "Birth Place: ~a\n\n" (alist-ref 'birthPlaceLabel result))))
102 results))))
103
104 ;; Test1: find URIs for Bill Gates and Microsoft; then list relationships
105 (def (test1)
106 (let* ((find-uris
107 (string-append
108 "PREFIX wd: <http://www.wikidata.org/entity/>\n"
109 "PREFIX wdt: <http://www.wikidata.org/prop/direct/>\n"
110 "PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n"
111 "SELECT ?bill ?microsoft WHERE {\n"
112 " ?bill rdfs:label \"Bill Gates\"@en .\n"
113 " ?bill wdt:P31 wd:Q5 .\n"
114 " ?microsoft rdfs:label \"Microsoft\"@en .\n"
115 ;; Ensure we pick the company entity
116 " ?microsoft wdt:P31/wdt:P279* wd:Q4830453 .\n"
117 "} LIMIT 1"))
118 (rows (query-wikidata/alist find-uris)))
119 (if (null? rows)
120 (display "No URIs found for Bill Gates/Microsoft.\n")
121 (let* ((row (car rows))
122 (bill (alist-ref 'bill row))
123 (microsoft (alist-ref 'microsoft row))
124 (rel-query
125 (string-append
126 "PREFIX wikibase: <http://wikiba.se/ontology#>\n"
127 "PREFIX bd: <http://www.bigdata.com/rdf#>\n"
128 "SELECT ?prop ?propLabel ?dir WHERE {\n"
129 " VALUES (?bill ?microsoft) { (<" bill "> <" microsoft ">) }\n"
130 " ?wdprop wikibase:directClaim ?prop .\n"
131 " { BIND(\"Bill->Microsoft\" AS ?dir) ?bill ?prop ?microsoft . }\n"
132 " UNION\n"
133 " { BIND(\"Microsoft->Bill\" AS ?dir) ?microsoft ?prop ?bill . }\n"
134 " SERVICE wikibase:label { bd:serviceParam wikibase:language \"en\". }\n"
135 "}\n"
136 "ORDER BY ?propLabel"))
137 (rels (query-wikidata/alist rel-query)))
138 (display (format "Bill Gates URI: ~a\n" bill))
139 (display (format "Microsoft URI: ~a\n" microsoft))
140 (if (null? rels)
141 (display "No direct relationships found.\n")
142 (for-each
143 (lambda (r)
144 (display (format "~a: ~a\n"
145 (alist-ref 'dir r)
146 (or (alist-ref 'propLabel r)
147 (alist-ref 'prop r)))))
148 rels))))))
149
150 ;; Test1 with User-Agent from env var WDQS_UA
151 (def (test1-ua)
152 (let* ((ua (or (getenv "WDQS_UA")
153 "YourApp/1.0 (https://your.site; you@site)"))
154 (find-uris
155 (string-append
156 "PREFIX wd: <http://www.wikidata.org/entity/>\n"
157 "PREFIX wdt: <http://www.wikidata.org/prop/direct/>\n"
158 "PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n"
159 "SELECT ?bill ?microsoft WHERE {\n"
160 " ?bill rdfs:label \"Bill Gates\"@en .\n"
161 " ?bill wdt:P31 wd:Q5 .\n"
162 " ?microsoft rdfs:label \"Microsoft\"@en .\n"
163 " ?microsoft wdt:P31/wdt:P279* wd:Q4830453 .\n"
164 "} LIMIT 1"))
165 (rows (query-wikidata/alist find-uris ua)))
166 (if (null? rows)
167 (display "No URIs found for Bill Gates/Microsoft.\n")
168 (let* ((row (car rows))
169 (bill (alist-ref 'bill row))
170 (microsoft (alist-ref 'microsoft row))
171 (rel-query
172 (string-append
173 "PREFIX wikibase: <http://wikiba.se/ontology#>\n"
174 "PREFIX bd: <http://www.bigdata.com/rdf#>\n"
175 "SELECT ?prop ?propLabel ?dir WHERE {\n"
176 " VALUES (?bill ?microsoft) { (<" bill "> <" microsoft ">) }\n"
177 " ?wdprop wikibase:directClaim ?prop .\n"
178 " { BIND(\"Bill->Microsoft\" AS ?dir) ?bill ?prop ?microsoft . }\n"
179 " UNION\n"
180 " { BIND(\"Microsoft->Bill\" AS ?dir) ?microsoft ?prop ?bill . }\n"
181 " SERVICE wikibase:label { bd:serviceParam wikibase:language \"en\". }\n"
182 "}\n"
183 "ORDER BY ?propLabel"))
184 (rels (query-wikidata/alist rel-query ua)))
185 (display (format "Bill Gates URI: ~a\n" bill))
186 (display (format "Microsoft URI: ~a\n" microsoft))
187 (if (null? rels)
188 (display "No direct relationships found.\n")
189 (for-each
190 (lambda (r)
191 (display (format "~a: ~a\n"
192 (alist-ref 'dir r)
193 (or (alist-ref 'propLabel r)
194 (alist-ref 'prop r)))))
195 rels))))))
196
197 ;; Example Usage with User-Agent from env var WDQS_UA
198 (def (test2-ua)
199 (let* ((ua (or (getenv "WDQS_UA")
200 "YourApp/1.0 (https://your.site; you@site)"))
201 (query
202 (string-append
203 "PREFIX wd: <http://www.wikidata.org/entity/>\n"
204 "PREFIX wdt: <http://www.wikidata.org/prop/direct/>\n"
205 "PREFIX wikibase: <http://wikiba.se/ontology#>\n"
206 "PREFIX bd: <http://www.bigdata.com/rdf#>\n"
207 "SELECT ?birthDate ?birthPlaceLabel WHERE {\n"
208 " wd:Q7249 wdt:P569 ?birthDate .\n"
209 " wd:Q7249 wdt:P19 ?birthPlace .\n"
210 " SERVICE wikibase:label { bd:serviceParam wikibase:language \"en\". }\n"
211 "}")))
212 (let ((results (query-wikidata/alist query ua)))
213 (for-each
214 (lambda (result)
215 (display (format "Birth Date: ~a\n" (alist-ref 'birthDate result)))
216 (display (format "Birth Place: ~a\n\n" (alist-ref 'birthPlaceLabel result))))
217 results))))
The core of this example is built around the query-wikidata and query-wikidata/alist functions. These procedures handle the low-level details of HTTP communication with the Wikidata SPARQL endpoint. They take a SPARQL query string, URI-encode it, and embed it into a GET request URL. They set a User-Agent string, a best practice that helps service operators identify the source of traffic; the functions allow this header to be overridden via an optional argument. After a successful request, the real data processing begins. The raw JSON response from a SPARQL endpoint is deeply nested, with each result variable wrapped in an object containing its type and value. The helper functions process-sparql-results and process-sparql-results-alist traverse this structure to extract the essential value of each binding, returning a clean list of results where each result is either a hash table or an association list mapping variable names (as symbols) to their values.
The included test functions, test1 and test2, serve as practical examples of this code’s capabilities. The test2 function is a straightforward lookup, retrieving the birth date and birthplace for a specific Wikidata entity (Grace Hopper, wd:Q7249). In contrast, test1 demonstrates a more powerful, dynamic pattern. It first runs a query to find the URIs for “Bill Gates” and “Microsoft” based on their English labels and types (human and business, respectively). It then uses these dynamically discovered URIs to construct a second query that finds all direct properties linking the two entities. This two-step approach is a common and robust method for interacting with linked data systems. Furthermore, the test1-ua and test2-ua variants illustrate how to provide a custom User-Agent string, for instance by reading it from an environment variable, showcasing the flexibility of the primary query functions.
Example Output
We use a Makefile target to start a Gerbil Scheme REPL with the example code loaded:
Here is the Makefile:
1 $ cat Makefile
2 run:
3 gxi -L wikidata.ss -
4
5 test:
6 gxi -L wikidata.ss -e "(test2)"
7
8 run-agent:
9 WDQS_UA="$${WDQS_UA:-YourApp/1.0 (https://your.site; you@site)}" gxi -L wikidata.ss -e "(test2-ua)"
10
11 test1:
12 gxi -L wikidata.ss -e "(test1)"
13
14 run-agent1:
15 WDQS_UA="$${WDQS_UA:-YourApp/1.0 (https://your.site; you@site)}" gxi -L wikidata.ss -e "(test1-ua)"
And sample output:
1 $ make run-agent1
2 WDQS_UA="${WDQS_UA:-YourApp/1.0 (https://your.site; you@site)}" gxi -L wikidata.ss -e "(test1-ua)"
3 Bill Gates URI: http://www.wikidata.org/entity/Q5284
4 Microsoft URI: http://www.wikidata.org/entity/Q2283
5 Microsoft->Bill: http://www.wikidata.org/prop/direct/P112
6 Bill->Microsoft: http://www.wikidata.org/prop/direct/P1830
7 $ make
8 gxi -L wikidata.ss -
9 > (test1)
10 Bill Gates URI: http://www.wikidata.org/entity/Q5284
11 Microsoft URI: http://www.wikidata.org/entity/Q2283
12 Microsoft->Bill: http://www.wikidata.org/prop/direct/P112
13 Bill->Microsoft: http://www.wikidata.org/prop/direct/P1830
14 > (test2)
15 Birth Date: 1862-02-14T00:00:00Z
16 Birth Place: Venice