Cover Material, Copyright, and License

This eBook will be updated occasionally so please periodically check the leanpub.com web page for this book for updates.

Please visit the author’s website.

If you found a copy of this book on the web and find it of value then please consider buying a copy at leanpub.com/haskell-cookbook to support the author and fund work for future updates.

",{"_434":435,"_44":45,"_48":49,"_46":47,"_66":67,"_436":437,"_536":432},"previewContent","

\n Cover Material, Copyright, and License\n
\n Preface\n
- \n A Request from the Author\n
- \n Structure of the Book\n
- \n Code Examples\n
- \n Functional Programming Requires a Different Mind Set\n
- \n eBooks Are Living Documents\n
- \n Setting Up Your Development Environment\n
- \n Why Haskell?\n
- \n Enjoy Yourself\n
- \n Acknowledgements\n
\n
\n Section 1 - Tutorial\n
\n Tutorial on Pure Haskell Programming\n
- \n Interactive GHCi Shell\n
- \n Introduction to Haskell Types\n
- \n Functions Are Pure\n
- \n Using Parenthesis or the Special $ Character and Operator Precedence\n
- \n Lazy Evaluation\n
- \n Understanding List Comprehensions\n
- \n Haskell Rules for Indenting Code\n
- \n Understanding let and where\n
- \n Conditional do Expressions and Anonymous Functions\n
- \n Maps\n
- \n Sets\n
- \n More on Functions\n
- \n Comments on Dealing With Immutable Data and How to Structure Programs\n
- \n Error Handling\n
- \n Testing Haskell Code\n
- \n Pure Haskell Wrap Up\n
\n
\n Tutorial on Impure Haskell Programming\n
- \n Hello IO () Monad\n
- \n A Note About >> and >>= Operators\n
- \n Console IO Example with Stack Configuration\n
- \n File IO\n
- \n Error Handling in Impure Code\n
- \n Network IO\n
- \n A Haskell Game Loop that Maintains State Functionally\n
- \n A More Detailed Look at Monads\n
- \n Using Applicative Operators <$> and <*>: Finding Common Words in Files\n
- \n List Comprehensions Using the do Notation\n
- \n Dealing With Time\n
- \n Using Debug.Trace\n
- \n Wrap Up\n
\n
\n Section 2 - Cookbook\n
\n Using the OpenAI Large Language Model APIs in Haskell\n
- \n Example Client Code\n
- \n Adding a Simple Application: Find Place Names in Input Text\n
\n
\n A Web Application For Using the Google Gemini APIs\n
- \n Web App Wrap Up and a Description of Scotty Web Framework\n
\n
\n Command Line Utility To Use the Google Gemini APIs\n
\n Command Line Tool to Use Local Ollama LLM Server\n
\n Using the Brave Search API\n
\n Text Processing\n
- \n CSV Spreadsheet Files\n
- \n JSON Data\n
- \n Cleaning Natural Language Text\n
\n
\n Natural Language Processing Tools\n
- \n Resolve Entities in Text to DBPedia URIs\n
- \n Bag of Words Classification Model\n
- \n Text Summarization\n
- \n Part of Speech Tagging\n
- \n Natural Language Processing Wrap Up\n
\n
\n Linked Data and the Semantic Web\n
- \n The SPARQL Query Language\n
- \n A Haskell HTTP Based SPARQL Client\n
- \n Querying Remote SPARQL Endpoints\n
- \n Linked Data and Semantic Web Wrap Up\n
\n
\n Web Scraping\n
- \n Web Scraping Wrap Up\n
\n
\n Using Relational Databases\n
- \n Database Access for Sqlite\n
- \n Database Access for Postgres\n
\n
\n Haskell Program to Play the Blackjack Card Game\n
\n Section 3 - Larger Projects\n
\n Knowledge Graph Creator\n
- \n Code Layout For the KGCreator Project and strategies for sharing Haskell code between projects\n
- \n The Main Event: Detecting Entities in Text\n
- \n Utility Code for Generating RDF\n
- \n Utility Code for Generating Cypher Input Data for Neo4J\n
- \n Top Level API Code for Handling Knowledge Graph Data Generation\n
- \n Wrap Up for Automating the Creation of Knowledge Graphs\n
\n
\n Hybrid Haskell and Python Natural Language Processing\n
- \n Example Use of the Haskell NLP Client\n
- \n Setting up the Python NLP Server\n
- \n Understanding the Haskell NLP Client Code\n
- \n Wrap Up for Using the Python SpaCy NLP Service\n
\n
\n Hybrid Haskell and Python For Coreference Resolution\n
- \n Installing the Python Coreference Server\n
- \n Understanding the Haskell Coreference Client Code\n
- \n Key Points\n
- \n Wrap Up for Using the Python Coreference NLP Service\n
\n
\n Book Wrap Up\n
- \n Using Haskell for Modern AI Applications\n
\n
\n Appendix A - Haskell Tools Setup\n
- \n stack\n
- \n Emacs Setup\n
- \n Do you want more of an IDE-like Development Environment?\n
- \n hlint\n
\n

Cover Material, Copyright, and License

This eBook will be updated occasionally so please periodically check the leanpub.com web page for this book for updates.

Please visit the author’s website.

If you found a copy of this book on the web and find it of value then please consider buying a copy at leanpub.com/haskell-cookbook to support the author and fund work for future updates.

Preface

This is the preface the newest version of this book released September 2024.

If you enjoy my eBooks and would like to work with me then please consider [hiring me for part time remote work] https://markwatson.com/#tip for part time consulting work.

It took me over a year learning Haskell before I became comfortable with the language because I tried to learn too much at once. There are two aspects to Haskell development: writing pure functional code and writing impure code that needs to maintain state and generally deal with the world non-deterministically. I usually find writing pure functional Haskell code to be easy and a lot of fun. Writing impure code is sometimes a different story. This is why I am taking a different approach to teaching you to program in Haskell: we begin techniques for writing concise, easy to read and understand efficient pure Haskell code. I will then show you patterns for writing impure code to deal with file IO, network IO, database access, and web access. You will see that the impure code tends to be (hopefully!) a small part of your application and is isolated in the impure main program and in a few impure helper functions used by the main program. Finally, we will look at a few larger Haskell programs:

\nknowledge_graph_creator helps to automate the process of creating Knowledge Graphs from raw text input and generates data for both the Neo4J open source graph database as well as RDF data for use in semantic web and linked data applications.\n
\nHybridHaskellPythonNlp is a hybrid project: a Python web service that provides access to the SpaCy natural language processing (NLP) library and select NLP deep learning models and a Haskell client for accessing this service.\n
\nHybridHaskellPythonCorefAnaphoraResolution that uses another deep learning model to replace pronouns in text with the original nouns that the pronouns refer to.\n
\nUse of Large Language Models (LLMs) form OpenAI.\n

I have also added chapters on using the OpenAI LLM APIs and the Brave search APIs.

A Request from the Author

I spent time writing this book to help you, dear reader. I release this book under the Creative Commons “share and share alike, no modifications, no commercial reuse” license and set the minimum purchase price to $7.99 in order to reach the most readers. You can also read this (and all of my eBooks) for free on leanpub.com.

If you would like to support my work please consider purchasing my books on Leanpub and star my git repositories that you find useful on GitHub. You can also interact with me on social media on X/Twitter.

I enjoy writing and your support helps me write new editions and updates for my books and to develop new book projects. Thank you!

Structure of the Book

The first section of this book contains two chapters:

\n
A tutorial on pure Haskell development: no side effects.
\n
\n
A tutorial on impure Haskell development: dealing with the world (I/O, network access, database access, etc.). This includes examples of file IO and network programming as well as writing short applications: a mixture of pure and impure Haskell code.
\n

After working through these tutorial chapters you will understand enough of Haskell development to understand and be able to make modifications for your own use of the cookbook examples in the second section. Some of the general topics will be covered again in the second book section that contains longer sample applications. For example, you will learn the basics for interacting with Sqlite and Postgres databases in the tutorial on impure Haskell code but you will see a much longer example later in the book when I provide code that implements a natural language processing (NLP) interface to relational databases.

The second section contains the following recipes implemented as complete programs:

\nUsing the OpenAI LLM APIs\n
\nUsing the Brave search APIs\n
\nTextprocessing CSV Files\n
\nTextprocessing JSON Files\n
\nNatural Language Processing (NLP) interface to relational databases, including annotating English text with Wikipedia/DBPedia URIs for entities in the original text. Entities can be people, places, organizations, etc.\n
\nAccessing and Using Linked Data\n
\nQuerying Semantic Web RDF Data Sources\n
\nWeb scraping data on web sites\n
\nUsing Sqlite and Postgres relational databases\n
\nPlay a simple version of Blackjack card game\n

Code Examples

The code examples in this book are licensed under two software licenses and you can choose the license that works best for your needs: Apache 2 and GPL 3. To be clear, you can use the examples in commercial projects under the Apache 2 license and if you like to write Free (Libre) software then use the GPL 3 license.

We will use stack as a build system for all code examples. The code examples are provided as 22 separate stack based projects. These examples are found on GitHub https://github.com/mark-watson/haskell_tutorial_cookbook_examples. Recently in 2024 I have also added instructions in the example code on GitHub to alternatively build and run the examples using Cabal.

Functional Programming Requires a Different Mind Set

You will learn to look at problems differently when you write functional programs. We will use a bottom up approach in most of the examples in this book. I like to start by thinking of the problem domain and decide how I can represent the data required for the problem at hand. I prefer to use native data structures. This is the opposite approach to object oriented development where considerable analysis effort and coding effort is required to define class hierachies to represent data. In most of the code we use simple native data types like lists and maps.

Once we decide how to represent data for a program we then start designing and implementing simple functions to operate on and transform data. If we find ourselves writing functions that are too long or too complex, we can break up code into simpler functions. Haskell has good language support for composing simple functions into more complex operations.

I have spent many years engaged in object oriented programming starting with CLOS for Common Lisp, C++, Java, and Ruby. I now believe that in general, and I know it is sometimes a bad idea to generalize too much, functional programming is a superior paradigm to object oriented programming. Convincing you of this belief is one of my goals in writing this book!

eBooks Are Living Documents

I wrote printed books for publishers like Springer-Verlag, McGraw-Hill, and Morgan Kaufman before I started self-publishing my own books. I prefer eBooks because I can update already published books and update the code examples for eBooks.

I encourage you to periodically check for free updates to both this book and the code examples on the leanpub.com web page for this book.

Setting Up Your Development Environment

I strongly recommend that you use the stack tool from the stack website. This web site has instructions for installing stack on OS X, Windows, and Linux. If you don’t have stack installed yet please do so now and follow the “getting started” instructions for creating a small project. Appendix A contains material to help get you set up.

It is important for you to learn the basics of using stack before jumping into this book because I have set up all of the example programs using stack.

The github repository for the examples in this book is located at github.com/mark-watson/haskell_tutorial_cookbook_examples.

Many of the example listings for code examples are partial or full listing of files in my github repository. I show the file name, the listing, and the output. To experiment with the example yourself you need to load it and execute the main function; for example, if the example file is TestSqLite1.hs in the sub-directory Database, then from the top level directory in the git repository for the book examples you would do the following:

\n\n

 1 $ haskell_tutorial_cookbook_examples git:(master) > cd Database \n 2 $ Database git:(master) > stack build --exec ghci\n 3 GHCi, version 7.10.3: http://www.haskell.org/ghc/  :? for help\n 4 Prelude> :l TestSqLite1\n 5 [1 of 1] Compiling Main             ( TestSqLite1.hs, interpreted )\n 6 Ok, modules loaded: Main.\n 7 *Main> main\n 8 \"Table names in database test.db:\"\n 9 \"test\"\n10 \"SQL to create table 'test' in database test.db:\"\n11 \"CREATE TABLE test (id integer primary key, str text)\"\n12 \"number of rows in table 'test':\"\n13 1\n14 \"rows in table 'test':\"\n15 (1,\"test string 2\")\n16 *Main>\n

\n\n

If you don’t want to run the example in a REPL in order to experiment with it interactively you can then just run it via stack using:

\n\n

1 $ Database git:(master) > stack build --exec TestSqlite1\n2 \"Table names in database test.db:\"\n3 \"test\"\n4 \"SQL to create table 'test' in database test.db:\"\n5 \"CREATE TABLE test (id integer primary key, str text)\"\n6 \"number of rows in table 'test':\"\n7 1\n8 \"rows in table 'test':\"\n9 (1,\"test string 2\")\n

\n\n

I include README.md files in the project directories with specific instructions.

I now use VSCode for most of my Haskell development. With the Haskell plugins VSCode offers auto-completion while typing and highlights syntax errors. Previously I use other editor for Haskell development. If you are an Emacs user I recommend that you follow the instructions in Appendix A, load the tutorial files into an Emacs buffer, build an example and open a REPL frame. If one is not already open type control-c control-l, switch to the REPL frame, and run the main function. When you make changes to the tutorial files, doing another control-c control-l will re-build the example in less than a second.

Appendix A also shows you how to setup the *stack Haskell build tool. As I update this book in September 2024 I have started using Cabal more often than Stack. The README files in the code examples for this book usually have both stack and Cabal instructions for building and how to run the examples.

Whether you use Emacs/VSCode or run a REPL in a terminal window (command window if you are using Windows) the important thing is to get used to and enjoy the interactive style of development that Haskell provides.

Why Haskell?

I have been using Lisp programming languages professionally since 1982. Lisp languages are flexible and appropriate for many problems. Some might disagree with me but I find that Haskell has most of the advantages of Lisp with the added benefit of being strongly typed. Both Lisp and Haskell support a style of development using an interactive shell (or “repl”).

What does being a strongly typed language mean? In a practical sense it means that you will often encounter syntax errors caused by type mismatches that you will need to fix before your code will compile (or run in the GHCi shell interpreter). Once your code compiles it will likely work, barring a logic error. The other benefit that you can get is having to write fewer unit tests - at least that is my experience. So, using a strongly typed language is a tradeoff. When I don’t use Haskell I tend to use dynamic languages like Common Lisp or Python.

Haskell supports type safety, concurrency, and high performance applications.

Enjoy Yourself

I have worked hard to make learning Haskell as easy as possible for you. If you are new to the Haskell programming language then I have something to ask of you, dear reader: please don’t rush through this book, rather take it slow and take time to experiment with the programming examples that most interest you.

Acknowledgements

I would like to thank my wife Carol Watson for editing the manuscript for this book. I would like to thank Roy Marantz, Michel Benard, Daniel Kroni, and David Jameson for reporting an errors.

Section 1 - Tutorial

The first section of this book contains two chapters:

\nA tutorial on pure Haskell development: no side effects.\n
\nA tutorial on impure Haskell development: dealing with the world (I/O, network access, database access, etc.)\n

After working through these two tutorial chapters you will have sufficient knowledge of Haskell development to understand the cookbook examples in the second section and be able to modify them for your own use. Some of the general topics will be covered again in the second book section that contains longer example programs.

Tutorial on Pure Haskell Programming

Pure Haskell code has no side effects and if written properly is easy to read and understand. I am assuming that you have installed stack using the directions in Appendix A. It is important to keep a Haskell interactive repl open as you read the material in this book and experiment with the code examples as you read. I don’t believe that you will be able to learn the material in this chapter unless you work along trying the examples and experimenting with them in an open Haskell repl!

The directory Pure in the git repository contains the examples for this chapter. Many of the examples contain a small bit of impure code in a main function. We will cover how this impure code works in the next chapter but let’s look at a short example of impure code that is contained inside a main function:

main = do\n  putStrLn (\"1 + 2 = \" ++ show (1 + 2))\n

\n\n

The function main is the entry point of this short two-line program. When the program is run, the main function will be executed.

Here the function main uses the do notation to execute a single IO action, but do can also execute a sequence of actions that can use impure Haskell code (i.e., code with side effects which we cover in the next chapter). The putStrLn function prints a string to the console. The printed string is constructed by concatenating three parts: “1 + 2 = “, the result of the expression 1 + 2 (which is 3), and the string representation of this result, which is obtained by calling the function show.

It’s worth noting that putStrLn writes a string to the standard output and also writes a new line character to the console. In general, the function show is used to convert any value to a string, here it is converting the result of 1+2 to string to concatenate it with the previous string.

Pure Haskell code performs no I/O, network access, access to shared in-memory data structures, etc.

The first time you build an example program with stack it may take a while since library dependencies need to be loaded from the web. In each example directory, after an initial stack build or stack ghci (to run the repl), then you should not notice this delay.

Interactive GHCi Shell

The interactive shell (often called a “repl”) is very useful for learning Haskell: understanding types and the value of expressions. While simple expressions can be typed directly into the GHCi shell, it is usually better to use an external text editor and load Haskell source files into the shell (repl). Let’s get started. Assuming that you have installed stack as described in Appendix A, please try:

~/$ cd haskell_tutorial_cookbook_examples/Pure\n~/haskell_tutorial_cookbook_examples/Pure$ stack ghci\nUsing main module: Package `Pure' component exe:Simple with main-is file: /home/mark\\\nw/BITBUCKET/haskell_tutorial_cookbook_examples/Pure/Simple.hs\nConfiguring GHCi with the following packages: Pure\nGHCi, version 7.10.3: http://www.haskell.org/ghc/  :? for help\n[1 of 1] Compiling Main             ( /home/markw/BITBUCKET/haskell_tutorial_cookboo\\\nk_examples/Pure/Simple.hs, interpreted )\nOk, modules loaded: Main.\n*Main> 1 + 2\n3\n*Main> (1 + 2)\n3\n*Main> :t (1 + 2)\n(1 + 2) :: Num a => a\n*Main> :l Simple.hs\n[1 of 1] Compiling Main             ( Simple.hs, interpreted )\nOk, modules loaded: Main.\n*Main> main\n1 + 2 = 3\n*Main> \n

\n\n

If you are working in a repl and edit a file you just loaded with :l, you can then reload the last file loaded using :r without specifying the file name. This makes it quick and easy to edit a Haskell file with an external editor like Emacs or Vi and reload it in the repl after saving changes to the current file.

Here we have evaluated a simple expression “1 + 2” in line 10. Notice that in line 12 we can always place parenthesis around an expression without changing its value. We will use parenthesis when we need to change the default orders of precedence of functions and operators and make the code more readable.

In line 14 we are using the ghci :t command to show the type of the expression (1 + 2). The type Num is a type class (i.e., a more general purpose type that other types can inherit from) that contains several sub-types of numbers. As examples, two subtypes of Num are Fractional (e.g., 3.5) and Integer (e.g., 123). Type classes provide a form of function overloading since existing functions can be redefined to handle arguments that are instances of new classes.

In line 16 we are using the ghci command :l to load the external file Simple.hs. This file contains a function called main so we can execute main after loading the file. The contents of Simple.hs is:

1 module Main where\n2 \n3 sum2 x y = x + y\n4 \n5 main = do\n6   putStrLn (\"1 + 2 = \" ++ show (sum2 1 2))\n

\n\n

Line 1 defines a module named Main. The rest of this file is the definition of the module. This form of the module do expression exports all symbols so other code loading this module has access to sum2 and main. The function sum2 takes two arguments, x and y, and returns their sum, x + y. If we only wanted to export main then we could use:

module Main (main) where\n

\n\n

The function sum2 takes two arguments and adds them together. I didn’t define the type of this function so Haskell does it for us using type inference.

*Main> :l Simple.hs \n[1 of 1] Compiling Main             ( Simple.hs, interpreted )\nOk, modules loaded: Main.\n*Main> :t sum2\nsum2 :: Num a => a -> a -> a\n*Main> sum2 1 2\n3\n*Main> sum2 1.0 2\n3.0\n*Main> :t 3.0\n3.0 :: Fractional a => a\n*Main> :t 3\n3 :: Num a => a\n*Main> (toInteger 3)\n3\n*Main> :t (toInteger 3)\n(toInteger 3) :: Integer\n*Main> \n

\n\n

What if you want to build a standalone executable program from the example in Simple.hs? Here is an example:

$ stack ghc Simple.hs\n[1 of 1] Compiling Main             ( Simple.hs, Simple.o )\nLinking Simple ...\n$ ./Simple \n1 + 2 = 3\n

\n\n

Most of the time we will use simple types built into Haskell: characters, strings, lists, and tuples. The type Char is a single character. One type of string is a list of characters [Char]. (Another type ByteString will be covered in later chapters.) Every element in a list must have the same type. A Tuple is like a list but elements can be different types. Here is a quick introduction to these types, with many more examples later:

 1 *Main> :t 's'\n 2 's' :: Char\n 3 *Main> :t \"tree\"\n 4 \"tree\" :: [Char]\n 5 *Main> 's' : \"tree\"\n 6 \"stree\"\n 7 *Main> :t \"tick\"\n 8 \"tick\" :: [Char]\n 9 *Main> 's' : \"tick\"\n10 \"stick\"\n11 *Main> :t [1,2,3,4]\n12 [1,2,3,4] :: Num t => [t]\n13 *Main> :t [1,2,3.3,4]\n14 [1,2,3.3,4] :: Fractional t => [t]\n15 *Main> :t [\"the\", \"cat\", \"slept\"]\n16 [\"the\", \"cat\", \"slept\"] :: [[Char]]\n17 *Main> [\"the\", \"cat\", \"slept\"] !! 0\n18 \"the\"\n19 *Main> head [\"the\", \"cat\", \"slept\"]\n20 \"the\"\n21 *Main> tail [\"the\", \"cat\", \"slept\"]\n22 [\"cat\",\"slept\"]\n23 *Main> [\"the\", \"cat\", \"slept\"] !! 1\n24 \"cat\"\n25 *Main> :t (20, 'c')\n26 (20, 'c') :: Num t => (t, Char)\n27 *Main> :t (30, \"dog\")\n28 (30, \"dog\") :: Num t => (t, [Char])\n29 *Main> :t (1, \"10 Jackson Street\", 80211, 77.5)\n30 (1, \"10 Jackson Street\", 80211, 77.5)\n31   :: (Fractional t2, Num t, Num t1) => (t, [Char], t1, t2)\n

\n\n

The GHCi repl command :t tells us the type of any expression or function. Much of your time developing Haskell will be spent with an open repl and you will find yourself checking types many times during a development session.

In line 1 you see that the type of ‘s’ is ‘s’ :: Char and in line 3 that the type of the string “tree” is [Char] which is a list of characters. The abbreviation String is defined for [Char]; you can use either. In line 9 we see the “cons” operator : used to prepend a character to a list of characters. The cons : operator works with all types contained in any lists. All elements in a list must be of the same type.

The type of the list of numbers [1,2,3,4] in line 11 is [1,2,3,4] :: Num t => [t]. The type Num is a general number type. The expression Num t => [t] is read as: “t is a type variable equal to Num and the type of the list is [t], or a list of Num values”. It bears repeating: all elements in a list must be of the same type. The functions head and tail used in lines 19 and 21 return the first element of a list and return a list without the first element.

You will use lists frequently but the restriction of all list elements being the same type can be too restrictive so Haskell also provides a type of sequence called tuple whose elements can be of different types as in the examples in lines 25-31.

Tuples of length 2 are special because functions fst and snd are provided to access the first and second pair value:

*Main> fst (1, \"10 Jackson Street\")\n1\n*Main> snd (1, \"10 Jackson Street\")\n\"10 Jackson Street\"\n*Main> :info fst\nfst :: (a, b) -> a  -- Defined in ‘Data.Tuple’\n*Main> :info snd\nsnd :: (a, b) -> b  -- Defined in ‘Data.Tuple’\n

\n\n

Please note that fst and snd will not work with tuples that are not of length 2. Also note that if you use the function length on a tuple, the result is always one because of the way tuples are defined as Foldable types, which we will use later.

Haskell provides a concise notation to get values out of long tuples. This notation is called destructuring:

1 *Main> let geoData = (1, \"10 Jackson Street\", 80211, 77.5)\n2 *Main> let (_,_,zipCode,temperature) = geoData \n3 *Main> zipCode \n4 80211\n5 *Main> temperature \n6 77.5\n

\n\n

Here, we defined a tuple geoData with values: index, street address, zip code, and temperature. In line two we extract the zip code and temperature. Another reminder: we use let in lines 1-2 because we are in a repl.

Like all programming languages, Haskell has operator precedence rules as these examples show:

 1 *Main> 1 + 2 * 10\n 2 21\n 3 *Main> 1 + (2 * 10)\n 4 21\n 5 *Main> length \"the\"\n 6 3\n 7 *Main> length \"the\" + 10\n 8 13\n 9 *Main> (length \"the\") + 10\n10 13\n

\n\n

The examples in lines 1-4 illustrate that the multiplication operator has a higher precedence than the addition operator.

*Main> :t length\nlength :: Foldable t => t a -> Int\n*Main> :t (+)\n(+) :: Num a => a -> a -> a\n

\n\n

Note that the function length starts with a lower case letter. All Haskell functions start with a lower case letter except for type constructor functions that we will get to later. A Foldable type can be iterated through and be processed with map functions (which we will use shortly).

We saw that the function + acts as an infix operator. We can convert infix functions to prefix functions by enclosing them in parenthesis:

*Main> (+) 1 2\n3\n*Main> div 10 3\n3\n*Main> 10 `div` 3\n3\n

\n\n

In this last example we also saw how a prefix function div can be used infix by enclosing it in back tick characters.

*Main> let x3 = [1,2,3]\n*Main> x3\n[1,2,3]\n*Main> let x4 = 0 : x3\n*Main> x4\n[0,1,2,3]\n*Main> x3 ++ x4\n[1,2,3,0,1,2,3]\n*Main> x4\n[0,1,2,3]\n*Main> x4 !! 0\n0\n*Main> x4 !! 100\n*** Exception: Prelude.!!: index too large\n*Main> let myfunc1 x y = x ++ y\n*Main> :t myfunc1\nmyfunc1 :: [a] -> [a] -> [a]\n*Main> myfunc1 x3 x4\n[1,2,3,0,1,2,3]\n

\n\n

Usually we define functions in files and load them as we need them. Here is the contents of the file myfunc1.hs:

1 myfunc1 :: [a] -> [a] -> [a]\n2 myfunc1 x y = x ++ y\n

\n\n

The first line is a type signature for the function and is not required; here the input arguments are two lists and the output is the two lists concatenated together. In line 1 note that a is a type variable that can represent any type. However, all elements in the two function input lists and the output list are constrained to be the same type.

*Main> :l myfunc1.hs \n[1 of 1] Compiling Main             ( myfunc1.hs, interpreted )\nOk, modules loaded: Main.\n*Main> myfunc1 [\"the\", \"cat\"] [\"ran\", \"up\", \"a\", \"tree\"]\n[\"the\",\"cat\",\"ran\",\"up\",\"a\",\"tree\"]\n

\n\n

Please note that the stack repl auto-completes using the tab character. For example, when I was typing in “:l myfunc1.hs” I actually just typed “:l myf” and then hit the tab character to complete the file name. Experiment with auto-completion, it will save you a lot of typing. In the following example, for instance, after defining the variable sentence I can just type “se” and the tab character to auto-complete the entire variable name:

*Main> let sentence = myfunc1 [\"the\", \"cat\"] [\"ran\", \"up\", \"a\", \"tree\"]\n*Main> sentence \n[\"the\",\"cat\",\"ran\",\"up\",\"a\",\"tree\"]\n

\n\n

The function head returns the first element in a list and the function tail returns all but the first elements in a list:

*Main> head sentence \n\"the\"\n*Main> tail sentence \n[\"cat\",\"ran\",\"up\",\"a\",\"tree\"]\n

\n\n

We can create new functions from existing arguments by supplying few arguments, a process known as “currying”:

*Main> let p1 = (+ 1)\n*Main> :t p1\np1 :: Num a => a -> a\n*Main> p1 20\n21\n

\n\n

In this last example the function + takes two arguments but if we only supply one argument a function is returned as the value: in this case a function that adds 1 to an input value.

We can also create new functions by composing existing functions using the infix function . that when placed between two function names produces a new function that combines the two functions. Let’s look at an example that uses . to combine the partial function (+ 1) with the function length:

*Main> let lengthp1 = (+ 1) . length\n*Main> :t lengthp1\nlengthp1 :: Foldable t => t a -> Int\n*Main> lengthp1 \"dog\"\n4\n

\n\n

Note the order of the arguments to the inline function .: the argument on the right side is the first function that is applied, then the function on the left side of the . is applied.

This is the second example where we have seen the type Foldable which means that a type can be mapped over, or iterated over. We will look at Haskell types in the next section.

Introduction to Haskell Types

This is a good time to spend more time studying Haskell types. We will see more material on Haskell types throughout this book so this is just an introduction using the data expression to define a Type MyColors defined in the file MyColors.hs:

data MyColors = Orange | Red | Blue | Green | Silver\n deriving (Show)\n

\n\n

This code defines a new data type in Haskell named MyColors that has five values: Orange, Red, Blue, Green or Silver. The keyword data is used to define a new data type, and the “|” symbol is used to separate the different possible values (also known as constructors) of the type.

The deriving (Show) clause at the end of the line tells the compiler to automatically generate an implementation of the Show type class for the MyColors type. In other words, we are asking the Haskell compiler to automatically generate a function show that can convert a value to a string. show is a standard function and in general we want it defined for all types. show converts an instance to a string value. This allows instances of MyColors to be converted to strings using the function show.

The MyColors type defined here is an enumeration (i.e., it is a fixed set of values), it’s an algebraic data type with no associated fields. This means that the type MyColors can only take one of the five values defined: Orange, Red, Blue, Green or Silver. There is another way to think about this. This code defines a new data type called MyColors with five constructors Orange, Red, Blue, Green or Silver.

Prelude> :l colors.hs \n[1 of 1] Compiling Main             ( colors.hs, interpreted )\nOk, modules loaded: Main.\n*Main> show Red\n\"Red\"\n*Main> let c1 = Green\n*Main> c1\nGreen\n*Main> :t c1\nc1 :: MyColors\n*Main> Red == Green\n\n<interactive>:60:5:\n    No instance for (Eq MyColors) arising from a use of ‘==’\n    In the expression: Red == Green\n    In an equation for ‘it’: it = Red == Green\n

\n\n

What went wrong here? The infix function == checks for equality and we did not define equality functions for our new type. Let’s fix the definition in the file colors.hs:

data MyColors = Orange | Red | Blue | Green | Silver\n deriving (Show, Eq)\n

\n\n

Because we are deriving Eq we are also asking the compiler to generate code to see if two instances of this class are equal. If we wanted to be able to order our colors then we would also derive Ord.

Now our new type has show, ==, and /= (inequality) defined:

Prelude> :l colors.hs \n[1 of 1] Compiling Main             ( colors.hs, interpreted )\nOk, modules loaded: Main.\n*Main> Red == Green\nFalse\n*Main> Red /= Green\nTrue\n

\n\n

Let’s also now derive Ord to have the compile generate a default function compare that operates on the type MyColors:

data MyColors = Orange | Red | Blue | Green | Silver\n deriving (Show, Eq, Ord)\n

\n\n

Because we are now deriving Ord the compiler will generate functions to calculate relative ordering for values of type MyColors. Let’s experiment with this:

*Main> :l MyColors.hs \n[1 of 1] Compiling Main             ( MyColors.hs, interpreted )\nOk, modules loaded: Main.\n*Main> :t compare\ncompare :: Ord a => a -> a -> Ordering\n*Main> compare Green Blue\nGT\n*Main> compare Blue Green\nLT\n*Main> Orange < Red\nTrue\n*Main> Red < Orange\nFalse\n*Main> Green < Red\nFalse\n*Main> Green < Silver\nTrue\n*Main> Green > Red\nTrue\n

\n\n

Notice that the compiler generates a compare function for the type MyColors that orders values by the order that they appear in the data expression. What if you wanted to order them in string sort order? This is very simple: we will remove Ord from the deriving clause and define our own function compare for type MyColors instead of letting the compiler generate it for us:

1 data MyColors = Orange | Red | Blue | Green | Silver\n2  deriving (Show, Eq)\n3           \n4 instance Ord MyColors where\n5   compare c1 c2 = compare (show c1) (show c2)\n

\n\n

In line 5 I am using the function show to convert instances of MyColors to strings and then the version of compare that is called in line 5 is the version the compiler wrote for us because we derived Show. Now the ordering is in string ascending sort order because we are using the compare function that is supplied for the type String:

*Main> :l MyColors.hs \n[1 of 1] Compiling Main             ( MyColors.hs, interpreted )\nOk, modules loaded: Main.\n*Main> Green > Red\nFalse\n

\n\n

Our new type MyColors is a simple type. Haskell also supports hierarchies of types called Type Classes and the type we have seen earlier Foldable is an example of a type class that other types can inherit from. For now, consider sub-types of Foldable to be collections like lists and trees that can be iterated over.

I want you to get in the habit of using :type and :info (usually abbreviated to :t and :i) in the GHCi repl. Stop reading for a minute now and type :info Ord in an open repl. You will get a lot of output showing you all of the types that Ord is defined for. Here is a small bit of what gets printed:

 1 *Main> :i Ord\n 2 class Eq a => Ord a where\n 3   compare :: a -> a -> Ordering\n 4   (<) :: a -> a -> Bool\n 5   (<=) :: a -> a -> Bool\n 6   (>) :: a -> a -> Bool\n 7   (>=) :: a -> a -> Bool\n 8   max :: a -> a -> a\n 9   min :: a -> a -> a\n10     -- Defined in ‘ghc-prim-0.4.0.0:GHC.Classes’\n11 instance Ord MyColors -- Defined at MyColors.hs:4:10\n12 instance (Ord a, Ord b) => Ord (Either a b)\n13   -- Defined in ‘Data.Either’\n14 instance Ord a => Ord [a]\n15   -- Defined in ‘ghc-prim-0.4.0.0:GHC.Classes’\n16 instance Ord Word -- Defined in ‘ghc-prim-0.4.0.0:GHC.Classes’\n17 instance Ord Ordering -- Defined in ‘ghc-prim-0.4.0.0:GHC.Classes’\n18 instance Ord Int -- Defined in ‘ghc-prim-0.4.0.0:GHC.Classes’\n19 instance Ord Float -- Defined in ‘ghc-prim-0.4.0.0:GHC.Classes’\n20 instance Ord Double -- Defined in ‘ghc-prim-0.4.0.0:GHC.Classes’\n

\n\n

Lines 1 through 8 show you that Ord is a subtype of Eq that defines functions compare, max, and min as well as the four operators <, <=, >=, and >=. When we customized the compare function for the type MyColors, we only implemented compare. That is all that we needed to do since the other operators rely on the implementation of compare.

Once again, I ask you to experiment with the example type MyColors in an open GHCi repl:

*Main> :t max\nmax :: Ord a => a -> a -> a\n*Main> :t Green\nGreen :: MyColors\n*Main> :i Green\ndata MyColors = ... | Green | ...   -- Defined at MyColors.hs:1:39\n*Main> max Green Red\nRed\n

\n\n

The following diagram shows a partial type hierarchy of a few types included in the standard Haskell Prelude (this is derived from the Haskell Report at haskell.org):

\n\n

\"Example — Figure 1. Example Haskell Type Hierarchy

Here you see that type Num and Ord are sub-types of type Eq, Real is a sub-type of Num, etc. We will see the types Monad and Functor in the next chapter.

Functions Are Pure

Again, it is worth pointing out that Haskell functions do not modify their inputs values. The common pattern is to pass immutable values to a function and modified values are returned. As a first example of this pattern we will look at the standard function map that takes two arguments: a function that converts a value of any type a to another type b, and a list of type a. Functions that take other functions as arguments are called higher order functions. The result is another list of the same length whose elements are of type b and the elements are calculated using the function passed as the first argument. Let’s look at a simple example using the function (+ 1) that adds 1 to a value:

*Main> :t map\nmap :: (a -> b) -> [a] -> [b]\n*Main> map (+ 1) [10,20,30]\n[11,21,31]\n*Main> map (show . (+ 1)) [10,20,30]\n[\"11\",\"21\",\"31\"]\n

\n\n

In the first example, types a and b are the same, a Num. The second example used a composed function that adds 1 and then converts the example to a string. Remember: the function show converts a Haskell data value to a string. In this second example types a and b are different because the function is mapping a number to a string.

The directory haskell_tutorial_cookbook_examples/Pure contains the examples for this chapter. We previously used the example file Simple.hs. Please note that in the rest of this book I will omit the git repository top level directory name haskell_tutorial_cookbook_examples and just specify the sub-directory name:

module Main where\n\nsum2 x y = x + y\n\nmain = do\n  putStrLn (\"1 + 2 = \" ++ show (sum2 1 2))\n

\n\n

Please note that the last two lines of this listing is not pure code since we are creating a side effect of printing data. We will cover impure code in the next chapter but for now we briefly review to do construct:

In Haskell, the do construct, especially when used with main = do, serves as a way to sequence multiple IO actions (operations that interact with the external world) in a clear and readable manner. It’s syntactic sugar that makes it easier to work with monads, particularly the IO monad.

Types of impure code allowed within a do block:

\nInput/Output (I/O): Reading from files (readFile), writing to files (writeFile, appendFile), interacting with the console (getLine, putStrLn), and network communication.\n
\nState Management: Using monads like State or ST to manage mutable state within a purely functional context.\n
\nExceptions: Handling runtime errors using functions like catch or try.\n
\nForeign Function Interface (FFI): Calling external C libraries or interacting with other languages.\n
\nConcurrency: Spawning threads or using other concurrency primitives.\n
\nTime-related operations: Getting the current time, introducing delays, or working with timeouts.\n
\nRandomness: Generating random numbers or performing other non-deterministic actions.\n

Essentially, any operation that has side effects or interacts with the world outside the pure functional realm of Haskell is considered impure and typically needs to be performed within the context of a do block or using other monadic constructs.

For now let’s just look at the mechanics of executing this file without using the REPL (started with stack ghci). We can simply build and run this example using stack, which is covered in some detail in Appendix A:

\n\n

stack build —exec Simple

 1 This command builds the project defined in the configuration files *Pure.cabal* and \\\n 2 *stack.yaml* (the format and use of these files is briefly covered in detail in Appe\n 3 ndix A and there is more [reference material here](https://docs.haskellstack.org/en/\n 4 stable/yaml_configuration/)). This example defines two functions: **sum2** and **mai\n 5 n**. **sum2** is a pure Haskell function with no state, no interaction with the outs\n 6 ide world like file IO, etc., and no non-determinism. **main** is an impure function\n 7 , and we will look at impure Haskell code in some detail in the next chapter. As you\n 8  might guess the output of this code snippet is\n 9 \n10 ```haskell{line-numbers: false}\n11 1 + 2 = 3\n

\n\n

To continue the tutorial on using pure Haskell functions, once again we will use stack to start an interactive repl during development:

1 markw@linux:~/haskell_tutorial_cookbook_examples/Pure$ stack ghci\n2 *Main> :t 3\n3 3 :: Num a => a\n4 *Main> :t \"dog\"\n5 \"dog\" :: [Char]\n6 *Main> :t main\n7 main :: IO ()\n8 *Main> \n

\n\n

In this last listing I don’t show the information about your Haskell environment and the packages that were loaded. In repl listings in the remainder of this book I will continue to edit out this Haskell environment information for brevity.

Line 4 shows the use of the repl shortcut :t to print out the type of a string which is an array of [Char], and the type of the function main is of type IO Action, which we will explain in the next chapter. An IO action contains impure code where we can read and write files, perform a network operation, etc. and we will look at IO Action in the next chapter.

Using Parenthesis or the Special $ Character and Operator Precedence

We will look at operator and function precedence and the use of the $ character to simplify using parenthesis in expessions. By the way, in Haskell there is not much difference between operators and function calls except operators like +, etc. which are by default infix while functions are usually prefix. So except for infix functions that are enclosed in backticks (e.g., 10 div 3) Haskell usually uses prefix functions: a function followed by zero or more arguments. You can also use $ that acts as an opening parenthesis with a not-shown closing parenthesis at the end of the current expression (which may be multi-line). Here are some examples:

*Main> print (3 * 2)\n6\n*Main> print $ 3 * 2\n6\n*Main> last (take 10 [1..])\n10\n*Main> last $ take 10 [1..]\n10\n*Main> ((take 10 [1..]) ++ (take 10 [1000..]))\n[1,2,3,4,5,6,7,8,9,10,1000,1001,1002,1003,1004,1005,1006,1007,1008,1009]\n*Main> take 10 [1..] ++ take 10 [1000..]\n[1,2,3,4,5,6,7,8,9,10,1000,1001,1002,1003,1004,1005,1006,1007,1008,1009]\n*Main> 1 + 2 * (4 * 5)\n41\n*Main> 2 * 3 + 10 * 30\n306\n

\n\n

I use the GHCi command :info (:i is an abbreviation) to check both operator precedence and the function signature if the operator is converted to a function by enclosing it in parenthesis:

*Main> :info *\nclass Num a where\n  ...\n  (*) :: a -> a -> a\n  ...\n    -- Defined in ‘GHC.Num’\ninfixl 7 *\n*Main> :info +\nclass Num a where\n  (+) :: a -> a -> a\n  ...\n    -- Defined in ‘GHC.Num’\ninfixl 6 +\n*Main> :info `div`\nclass (Real a, Enum a) => Integral a where\n  ...\n  div :: a -> a -> a\n  ...\n    -- Defined in ‘GHC.Real’\ninfixl 7 `div`\n*Main> :i +\nclass Num a where\n  (+) :: a -> a -> a\n  ...\n    -- Defined in ‘GHC.Num’\ninfixl 6 +\n

\n\n

Notice how + has lower precedence than *.

Just to be clear, understand how operators are used as functions and also how functions can be used as infix operators:

*Main> 2 * 3\n6\n*Main> (*) 2 3\n6\n*Main> 10 `div` 3\n3\n*Main> div 10 3\n3\n

\n\n

Especially when you are just starting to use Haskell it is a good idea to also use :info to check the type signatures of standard functions that you use. For example:

*Main> :info last\nlast :: [a] -> a    -- Defined in ‘GHC.List’\n*Main> :info map\nmap :: (a -> b) -> [a] -> [b]   -- Defined in ‘GHC.Base’\n

\n\n

Lazy Evaluation

Haskell is referred to as a lazy language.

In lazy evaluation, expressions are not evaluated until their results are absolutely necessary for the program’s execution. This contrasts with eager evaluation, where expressions are evaluated as soon as they are encountered, regardless of whether their values are immediately needed.

Benefits of Lazy Evaluation

\nEfficiency: By delaying computations until necessary, lazy evaluation can avoid performing unnecessary work, potentially leading to more efficient programs.\n
\nFlexibility: Lazy evaluation allows you to work with infinite data structures and perform operations on them without worrying about the entire structure being computed at once. Only the parts that are actually accessed will be evaluated.\n

Consider the following example:

 1 $ stack ghci\n 2 *Main> [0..10]\n 3 [0,1,2,3,4,5,6,7,8,9,10]\n 4 *Main> take 11 [0..]\n 5 [0,1,2,3,4,5,6,7,8,9,10]\n 6 *Main> let xs = [0..]\n 7 *Main> :sprint xs\n 8 xs = _\n 9 *Main> take 5 xs\n10 [0,1,2,3,4]\n11 *Main> :sprint xs\n12 xs = _\n13 *Main> \n

\n\n

In line 2 we are creating a list with 11 elements. In line 4 we are doing two things:

\nCreating an infinitely long list containing ascending integers starting with 0.\n
\nFetching the first 11 elements of this infinitely long list. It is important to understand that in line 4 only the first 11 elements are generated because that is all the take function requires.\n

In line 6 we are assigning another infinitely long list to the variable xs but the value of xs is unevaluated and a placeholder is stored to calculate values as required. In line 7 we use GHCi’s :sprint command to show a value without evaluating it. The output in line 8 _ indicated that the expression has yet to be evaluated.

Lines 9 through 12 remind us that Haskell is a functional language: the take function used in line 9 does not change the value of its argument so xs as seen in lines 10 and 12 is still unevaluated.

Understanding List Comprehensions

Effectively using list comprehensions makes your code shorter, easier to understand, and easier to maintain. Let’s start out with a few GHCi repl examples. You will learn a new GHCi repl trick in this section: entering multiple line expressions by using :{ and :} to delay evaluation until an entire expression is entered in the repl (listings in this section are reformatted to fit the page width):

1 *Main> [x | x <- [\"cat\", \"dog\", \"bird\"]]\n2 [\"cat\",\"dog\",\"bird\"]\n3 *Main> :{\n4 *Main| [(x,y) | x <- [\"cat\", \"dog\", \"bird\"],\n5 *Main|          y <- [1..2]]\n6 *Main| :}\n7 [(\"cat\",1),(\"cat\",2),(\"dog\",1),(\"dog\",2),(\"bird\",1),(\"bird\",2)]\n

\n\n

The list comprehension on line 1 assigns the elements of the list [“cat”, “dog”, “bird”] one at a time to the variable x and then collects all these values of x in a list value that is the value of the list comprehension. The list comprehension in line 1 is hopefully easy to understand but when we bind and collect multiple variables the situation, as seen in the example in lines 4 and 5, is not as easy to understand. The thing to remember is that the first variable gets iterated as an “outer loop” and the second variable is iterated as the “inner loop.” List comprehensions can use many variables and the iteration ordering rule is the same: last variable iterates first, etc.

*Main> :{\n*Main| [(x,y) | x <- [0..3],\n*Main|          y <- [1,3..10]]\n*Main| :}\n[(0,1),(0,3),(0,5),(0,7),(0,9),(1,1),(1,3),(1,5),(1,7),\n (1,9),(2,1),(2,3),(2,5),(2,7),(2,9),(3,1),(3,3),(3,5),\n (3,7),(3,9)]\n*Main> [1,3..10]\n[1,3,5,7,9]\n

\n\n

In this last example we are generating all combinations of [0..3] and [1,3..10] and storing the combinations as two element tuples. You could also store then as lists:

*Main> [[x,y] | x <- [1,2], y <- [10,11]]\n[[1,10],[1,11],[2,10],[2,11]]\n

\n\n

List comprehensions can also contain filtering operations. Here is an example with one filter:

*Main> :{\n*Main| [(x,y) | x <- [\"cat\", \"dog\", \"bird\"],\n*Main|          y <- [1..10],\n*Main|          y `mod` 3 == 0]\n*Main| :}\n[(\"cat\",3),(\"cat\",6),(\"cat\",9),\n (\"dog\",3),(\"dog\",6),(\"dog\",9),\n (\"bird\",3),(\"bird\",6),(\"bird\",9)]\n

\n\n

Here is a similar example with two filters (we are also filtering out all possible values of x that start with the character ‘d’):

*Main> :{\n*Main| [(x,y) | x <- [\"cat\", \"dog\", \"bird\"],\n*Main|          y <- [1..10],\n*Main|          y `mod` 3 == 0,\n*Main|          x !! 0 /= 'd']\n*Main| :}\n[(\"cat\",3),(\"cat\",6),(\"cat\",9),(\"bird\",3),(\"bird\",6),(\"bird\",9)]\n

\n\n

For simple filtering cases I usually use the filter function but list comprehensions are more versatile. List comprehensions are extremely useful - I use them frequently.

Lists are instances of the class Monad that we will cover in the next chapter (check out the section “List Comprehensions Using the do Notation”).

List comprehensions are powerful. I would like to end this section with another trick that does not use list comprehensions for building lists of tuple values: using the zip function:

*Main> let animals = [\"cat\", \"dog\", \"bird\"]\n*Main> zip [1..] animals\n[(1,\"cat\"),(2,\"dog\"),(3,\"bird\")]\n*Main> :info zip\nzip :: [a] -> [b] -> [(a, b)]   -- Defined in ‘GHC.List’\n

\n\n

The function zip is often used in this way when we have a list of objects and we want to operate on the list while knowing the index of each element.

Haskell Rules for Indenting Code

When a line of code is indented relative to the previous line of code, or several lines of code with additional indentation, then the indented lines act as if they were on the previous line. In other words, if code that should all be on one line must be split to multiple lines, then use indentation as a signal to the Haskell compiler.

Indentation of continuation lines should be uniform, starting in the same column. Here are some examples of good code, and code that will not compile:

  let a = 1    -- good \n      b = 2    -- good\n      c = 3    -- good\n      \n  let\n    a = 1        -- good \n    b = 2        -- good\n    c = 3        -- good\n  in a + b + c   -- good\n\n  let a = 1    -- will not compile (bad) \n       b = 2   -- will not compile (bad)\n      c = 3    -- will not compile (bad)\n      \n  let\n      a = 1    -- will not compile (bad) \n      b = 2    -- will not compile (bad)\n     c = 3     -- will not compile (bad)\n\n  let {\n      a = 1;    -- compiles but bad style (good) \n        b = 2;  -- compiles but bad style (good)\n      c = 3;    -- compiles but bad style (good)\n    } \n

\n\n

If you use C style braces and semicolons to mark end of expressions, then indenting does not matter as seen in lines 20 through 24. Otherwise, uniform indentation is a hint to the compiler.

The same indenting rules apply to other types of do expressions which we will see throughout this book for do, if, and other types of do expressions.

Understanding let and where\n

At first glance, let and where seem very similar in that they allow us to create temporary variables used inside functions. As the examples in the file LetAndWhere.hs show, there are important differences.

In the following code notice when we use let in pure code inside a function, we then use in to indicate the start of an expression to be evaluated that uses any variables defined in a let expression. Inside a do code block the in token is not needed and will cause a parse error if you use it. do code blocks are a syntactic sugar for use in impure Haskell code and we will use it frequently later in the book.

You also do not use in inside a list comprehension as seen in the function testLetComprehension in the next code listing:

 1 module Main where\n 2 \n 3 funnySummation w x y z =\n 4   let bob = w + x\n 5       sally = y + z\n 6   in bob + sally\n 7 \n 8 testLetComprehension =\n 9   [(a,b) | a <- [0..5], let b = 10 * a]\n10 \n11 testWhereBlocks a =\n12   z * q\n13     where\n14       z = a + 2\n15       q = 2\n16 \n17 functionWithWhere n  =\n18   (n + 1) * tenn\n19   where\n20     tenn = 10 * n\n21           \n22 main = do\n23   print $ funnySummation 1 2 3 4\n24   let n = \"Rigby\"\n25   print n\n26   print testLetComprehension\n27   print $ testWhereBlocks 11\n28   print $ functionWithWhere 1\n

\n\n

Compare the let do expressions starting on line 4 and 24. The first let occurs in pure code and uses in to define one or more do expressions using values bound in the let. In line 24 we are inside a monad, specifically using the do notation and here let is used to define pure values that can be used later in the do do expression.

Loading the last code example and running the main function produces the following output:

1 *Main> :l LetAndWhere.hs \n2 [1 of 1] Compiling Main             ( LetAndWhere.hs, interpreted )\n3 Ok, modules loaded: Main.\n4 *Main> main\n5 10\n6 \"Rigby\"\n7 [(0,0),(1,10),(2,20),(3,30),(4,40),(5,50)]\n8 26\n9 20\n

\n\n

This output is self explanatory except for line 7 that is the result of calling testLetComprehension that returns an example list comprehension [(a,b)|a<-[0..5],letb=10*a]

Conditional do Expressions and Anonymous Functions

The examples in the next three sub-sections can be found in haskell_tutorial_cookbook_examples/Pure/Conditionals.hs. You should read the following sub-sections with this file loaded (some GHCi repl output removed for brevity):

haskell_tutorial_cookbook_examples/Pure$ stack ghci\n*Main> :l Conditionals.hs \n[1 of 1] Compiling Main             ( Conditionals.hs, interpreted )\nOk, modules loaded: Main.\n*Main> \n

\n\n

Simple Pattern Matching

We previously used the built-in functions head that returns the first element of a list and tail that returns a list with the first element removed. We will define these functions ourselves using what is called wild card pattern matching. It is common to append the single quote character ’ to built-in functions when we redefine them so we name our new functions head’ and tail’. Remember when we used destructuring to access elements of a tuple? Wild card pattern matching is similar:

head'(x:_)  = x\ntail'(_:xs) = xs\n

\n\n

The underscore character _ matches anything and ignores the matched value. Our head and tail definitions work as expected:

*Main> head' [\"bird\",\"dog\",\"cat\"]\n\"bird\"\n*Main> tail' [0,1,2,3,4,5]\n[1,2,3,4,5]\n*Main> :type head'\nhead' :: [t] -> t\n*Main> :t tail'\ntail' :: [t] -> [t]\n

\n\n

Of course we frequently do not want to ignore matched values. Here is a contrived example that expects a list of numbers and doubles the value of each element. As for all of the examples in this chapter, the following function is pure: it can not modify its argument(s) and always returns the same value given the same input argument(s):

doubleList [] = []\ndoubleList (x:xs) = (* 2) x : doubleList xs\n

\n\n

In line 1 we start by defining a pattern to match the empty list. It is necessary to define this terminating condition because we are using recursion in line 2 and eventually we reach the end of the input list and make the recursive call doubleList []. If you leave out line 1 you then will see a runtime error like “Non-exhaustive patterns in function doubleList.” As a Haskell beginner you probably hate Haskell error messages and as you start to write your own functions in source files and load them into a GHCi repl or compile them, you will initially probably hate compilation error messages also. I ask you to take on faith a bit of advice: Haskell error messages and warnings will end up saving you a lot of effort getting your code to work properly. Try to develop the attitude “Great! The Haskell compiler is helping me!” when you see runtime errors and compiler errors.

In line 2 notice how I didn’t need to use extra parenthesis because of the operator and function application precedence rules.

*Main> doubleList [0..5]\n[0,2,4,6,8,10]\n*Main> :t doubleList \ndoubleList :: Num t => [t] -> [t]\n

\n\n

This function doubleList seems very unsatisfactory because it is so specific. What if we wanted to triple or quadruple the elements of a list? Do we want to write two new functions? You might think of adding an argument that is the multiplier like this:

bumpList n [] = []\nbumpList n (x:xs) = n * x : bumpList n xs\n

\n\n

is better, being more abstract and more general purpose. However, we will do much better.

Before generalizing the list manipulation process further, I would like to make a comment on coding style, specifically on not using unneeded parenthesis. In the last example defining bumpList if you have superfluous parenthesis like this:

bumpList n (x:xs) = (n * x) : bumpList (n xs)\n

\n\n

then the code still works correctly and is fairly readable. I would like you to get in the habit of avoiding extra unneeded parenthesis and one tool for doing this is running hlint (installing hlint is covered in Appendix A) on your Haskell code. Using hlint source file will provide warnings/suggestions like this:

haskell_tutorial_cookbook_examples/Pure$ hlint Conditionals.hs\nConditionals.hs:7:21: Warning: Redundant bracket\nFound:\n  ((* 2) x) : doubleList (xs)\nWhy not:\n  (* 2) x : doubleList (xs)\n\nConditionals.hs:7:43: Error: Redundant bracket\nFound:\n  (xs)\nWhy not:\n  xs\n

\n\n

hlint is not only a tool for improving your code but also for teaching you how to better program using Haskell. Please note that hlint provides other suggestions for Conditionals.hs that I am ignoring that mostly suggest that I replace our mapping operations with using the built-in map function and use functional composition. The sample code is specifically to show examples of pattern matching and is not as concise as it could be.

Are you satisfied with the generality of the function bumpList? I hope that you are not! We should write a function that will apply an arbitrary function to each element of a list. We will call this function map’ to avoid confusing our map’ function with the built-in function map.

The following is a simple implementation of a map function (we will see Haskell’s standard map functions in the next section):

 1 map' f [] = []\n 2 map' f (x:xs) = f x : map' f xs\n 3 ~~~~~~~~\n 4 \n 5 In line 2 we do not need parenthesis around **f x** because function application has\\\n 6  a higher precedence than the operator **:** which adds an element to the beginning \n 7 of a list.\n 8 \n 9 Are you pleased with how concise this definition of a map function is? Is concise co\\\n10 de like **map'** readable to you? Speaking as someone who has written hundreds of th\n11 ousands of lines of Java code for customers, let me tell you that I love the concise\n12 ness and readability of Haskell! I appreciate the Java ecosystem with many useful li\n13 braries and frameworks and augmented like fine languages like Clojure and JRuby, but\n14  in my opinion using Haskell is a more enjoyable and generally more productive langu\n15 age and programming environment.\n16 \n17 Let's experiment with our **map'** function:\n18 \n19 ```haskell{line-numbers: true}\n20 *Main> map' (* 7) [0..5]\n21 [0,7,14,21,28,35]\n22 *Main> map' (+ 1.1) [0..5]\n23 [1.1,2.1,3.1,4.1,5.1,6.1]\n24 *Main>  map' (\\x -> (x + 1) * 2) [0..5]\n25 [2,4,6,8,10,12]\n

\n\n

Lines 1 and 3 should be understandable to you: we are creating a partial function like ( 7)* and passing it to map’ to apply to the list [0..5].

The syntax for the function in line 5 is called an anonymous function. Lisp programers, like myself, refer to this as a lambda expression. In any case, I often prefer using anonymous functions when a function will not be used elsewhere. In line 5 the argument to the anonymous inline function is x and the body of the function is (x + 1) * 2.

I do ask you to not get carried away with using too many anonymous inline functions because they can make code a little less readable. When we put our code in modules, by default every symbol (like function names) in the module is externally visible. However, if we explicitly export symbols in a module do expression then only the explicitly exported symbols are visible by other code that uses the module. Here is an example:

module Test2 (doubler) where\n\nmap' f [] = []\nmap' f (x:xs) = (f x) : map' f xs\n\ntestFunc x = (x + 1) * 2\n\ndoubler xs = map' (* 2) xs\n

\n\n

In this example map’ and testFunc are hidden: any other module that imports Test2 only has access to doubler. It might help for you to think of the exported functions roughly as an interface for a module.

Pattern Matching With Guards

We will cover two important concepts in this section: using guard pattern matching to make function definitions shorter and easier to read and we will look at the Maybe type and how it is used. The Maybe type is mostly used in non-pure Haskell code and we will use it heavily later. The Maybe type is a Monad (covered in the next chapter). I introduce the Maybe type here since its use fits naturally with guard patterns.

Guards are more flexible than the pattern matching seen in the last section. I use pattern matching for simple cases of destructuring data and guards when I need the flexibility. You may want to revisit the examples in the last section after experimenting with and understanding the examples seen here.

The examples for this section are in the file Guards.hs. As a first simple example we will implement the Ruby language “spaceship operator”:

1 spaceship n\n2   | n < 0     = -1\n3   | n == 0    = 0\n4   | otherwise = 1\n

\n\n

Notice on line 1 that we do not use an = in the function definition when using guards. Each guard starts with |, contains a condition, and a value on the right side of the = sign.

1 *Main> spaceship (-10)\n2 -1\n3 *Main> spaceship 0\n4 0\n5 *Main> spaceship 17\n6 1\n

\n\n

Remember that a literal negative number as seen in line 1 must be wrapped in parenthesis, otherwise the Haskell compiler will interpret - as an operator.

Case Expressions

Case do expressions match a value against a list of possible values. It is common to use the wildcard matching value _ at the end of a case expression which can be of any type. Here is an example in the file Cases.hs:

 1 module Main where\n 2 \n 3 numberOpinion n = \n 4   case n of\n 5     0 -> \"Too low\"\n 6     1 -> \"just right\"\n 7     _ -> \"OK, that is a number\"\n 8     \n 9 main = do\n10   print $ numberOpinion 0\n11   print $ numberOpinion 1\n12   print $ numberOpinion 2\n

\n\n

The code in lines 3-7 defines the function numberOpinion that takes a single argument “n”. We use a case expression to match the value of n against several possible cases. Each of these cases is defined using the -> operator, followed by an expression to be evaluated if the case is matched.

The first case, 0 -> ‘Too low’ matches the value of n against 0, if the value of “n” is 0, the function will return the string “Too low”. The second case, 1 -> ‘just right’ matches the value of n against 1, if the value of n is 1, the function will return the string “just right”.\nThe last case is different in that it is a catch all case using the _ as a wild card match. So, _ -> ‘OK, that is a number’ matches any other values of n: if the value of nn is not 0 or 1 the function will return the string “OK, that is a number.”

If Then Else expressions

Haskell has if then else syntax built into the language - if is not defined as a function. Personally I do not use if then else in Haskell very often. I mostly use simple pattern matching and guards. Here are some short examples from the file IfThenElses.hs:

ageToString age =\n  if age < 21 then \"minor\" else \"adult\"\n

\n\n

All if statements must have both a then expression and a else expression.

haskell_tutorial_cookbook_examples/Pure$ stack ghci\n*Main> :l IfThenElses.hs \n[1 of 1] Compiling Main             ( IfThenElses.hs, interpreted )\nOk, modules loaded: Main.\n*Main> ageToString 15\n\"minor\"\n*Main> ageToString 37\n\"adult\"\n

\n\n

Maps

Maps are simple to construct using a list of key-value tuples and are by default immutable. There is an example using mutable maps in the next chapter.

We will look at the module Data.Map first in a GHCi repl, then later in a few full code examples. There is something new in line 1 of the following listing: I am assigning a short alias M to the module Data.Map. In referencing a function like fromList (which converts a list of tuples to a map) in the Data.Map module I can use M.fromList instead of Data.Map.fromList. This is a common practice so when you read someone else’s Haskell code, one of the first things you should do when reading a Haskell source file is to make note of the module name abbreviations at the top of the file.

haskell_tutorial_cookbook_examples/Pure$ stack ghci\n*Main> import qualified Data.Map as M\n*Main M> :t M.fromList\nM.fromList :: Ord k => [(k, a)] -> M.Map k a\n*Main M> let aTestMap = M.fromList [(\"height\", 120), (\"weight\", 15)]\n*Main M> :t aTestMap \naTestMap :: Num a => M.Map [Char] a\n*Main M> :t lookup\nlookup :: Eq a => a -> [(a, b)] -> Maybe b\n*Main M> :t M.lookup\nM.lookup :: Ord k => k -> M.Map k a -> Maybe a\n*Main M> M.lookup \"weight\" aTestMap \nJust 15\n*Main M> M.lookup \"address\" aTestMap \nNothing\n

\n\n

The keys in a map must all be the same type and the values are also constrained to be of the same type. I almost always create maps using the helper function fromList in the module Data.Maps. We will only be using this method of map creation in later examples in this book so I am skipping coverage of other map building functions. I refer you to the Data.Map documentation.

The following example shows one way to use the Just and Nothing return values:

module MapExamples where\n\nimport qualified Data.Map as M -- from library containers\n\naTestMap = M.fromList [(\"height\", 120), (\"weight\", 15)]\n\ngetNumericValue key aMap =\n  case M.lookup key aMap of\n    Nothing -> -1\n    Just value -> value\n\nmain = do\n  print $ getNumericValue \"height\" aTestMap\n  print $ getNumericValue \"age\" aTestMap\n

\n\n

The function getNumericValue shows one way to extract a value from an instance of type Maybe. The function lookup returns a Maybe value and in this example I use a case statement to test for a Nothing value or extract a wrapped value in a Just instance. Using Maybe in Haskell is a better alternative to checking for null values in C or Java.

The output from running the main function in module MapExamples is:

haskell_tutorial_cookbook_examples/Pure$ stack ghci\n*Main> :l MapExamples.hs \n[1 of 1] Compiling MapExamples      ( MapExamples.hs, interpreted )\nOk, modules loaded: MapExamples.\n*MapExamples> main\n120\n-1\n

\n\n

Sets

The documentation of Data.Set.Class can be found here and contains overloaded functions for the types of sets defined here.

For most of my work and for the examples later in this book, I create immutable sets from lists and the only operation I perform is checking to see if a value is in the set. The following examples in GHCI repl are what you need for the material in this book:

*Main> import qualified Data.Set as S \n*Main S> let testSet = S.fromList [\"cat\",\"dog\",\"bird\"]\n*Main S> :t testSet\ntestSet :: S.Set [Char]\n*Main S> S.member \"bird\" testSet\nTrue\n*Main S> S.member \"snake\" testSet\nFalse\n

\n\n

Sets and Maps are immutable so I find creating maps using a lists of key-value tuples and creating sets using lists is fine. That said, coming from the mutable Java, Ruby, Python, and Lisp programming languages, it took me a while to get used to immutability in Haskell.

More on Functions

In this section we will review what you have learned so far about Haskell functions and then look at a few more complex examples.

We have been defining and using simple functions and we have seen that operators behave like infix functions. We can make operators act as prefix functions by wrapping them in parenthesis:

*Main> 10 + 1\n11\n*Main> (+) 10 1\n11\n

\n\n

and we can make functions act as infix operators:

*Main> div 100 9\n11\n*Main> 100 `div` 9\n11\n

\n\n

This back tick function to operator syntax works with functions we write also:

*Main> let myAdd a b = a + b\n*Main> :t myAdd \nmyAdd :: Num a => a -> a -> a\n*Main> myAdd 1 2\n3\n*Main> 1 `myAdd` 2\n3\n

\n\n

Because we are working in a GHCi repl, in line 1 we use let to define the function myAdd. If you defined this function in a file and then loaded it, you would not use a let.

In the map examples where we applied a function to a list of values, so far we have used functions that map input values to the same return type, like this (using both partial function evaluation and anonymous inline function):

*Main> map (* 2) [5,6]\n[10,12]\n*Main> map (\\x -> 2 * x) [5,6]\n[10,12]\n

\n\n

We can also map to different types; in this example we map from a list of Num values to a list containing sub-lists of Num values:

*Main> let makeList n = [0..n]\n*Main> makeList 3\n[0,1,2,3]\n*Main> map makeList [2,3,4]\n[[0,1,2],[0,1,2,3],[0,1,2,3,4]]\n

\n\n

As usual, I recommend that when you work in a GHCi repl you check the types of functions and values you are working with:

1 *Main> :t makeList \n2 makeList :: (Enum t, Num t) => t -> [t]\n3 *Main> :t [1,2]\n4 [1,2] :: Num t => [t]\n5 *Main> :t [[0,1,2],[0,1,2,3],[0,1,2,3,4]]\n6 [[0,1,2],[0,1,2,3],[0,1,2,3,4]] :: Num t => [[t]]\n7 *Main> \n

\n\n

In line 2 we see that for any type t the function signature is t -> [t] where the compiler determines that t is constrained to be a Num or Enum by examining how the input variable is used as a range parameter for constructing a list. Let’s make a new function that works on any type:

 1 *Main> let make3 x = [x,x,x]\n 2 *Main> :t make3\n 3 make3 :: t -> [t]\n 4 *Main> :t make3 \"abc\"\n 5 make3 \"abc\" :: [[Char]]\n 6 *Main> make3 \"abc\"\n 7 [\"abc\",\"abc\",\"abc\"]\n 8 *Main> make3 7.1\n 9 [7.1,7.1,7.1]\n10 *Main> :t make3 7.1\n11 make3 7.1 :: Fractional t => [t]\n

\n\n

Notice in line 3 that the function make3 takes any type of input and returns a list of elements the same type as the input. We used makes3 both with a string argument and a fractional (floating point) number) argument.

Comments on Dealing With Immutable Data and How to Structure Programs

If you program in other programming languages that use mutable data then expect some feelings of disorientation initially when starting to use Haskell. It is common in other languages to maintain the state of a computation in an object and to mutate the value(s) in that object. While I cover mutable state in the next chapter the common pattern in Haskell is to create a data structure (we will use lists in examples here) and pass it to functions that return a new modified copy of the data structure as the returned value from the function. It is very common to keep passing the modified new copy of a data structure through a series of function calls. This may seem cumbersome when you are starting to use Haskell but quickly feels natural.

The following example shows a simple case where a list is constructed in the function main and passed through two functions doubleOddElements and times10Elements:

 1 module ChainedCalls where\n 2   \n 3 doubleOddElements =\n 4   map (\\x -> if x `mod` 2 == 0 then x else 2 * x)\n 5 \n 6 times10Elements = map (* 10)\n 7     \n 8 main = do\n 9   print $ doubleOddElements [0,1,2,3,4,5,6,7,8]\n10   let aList = [0,1,2,3,4,5]\n11   let newList = times10Elements $ doubleOddElements aList\n12   print newList\n13   let newList2 = (times10Elements . doubleOddElements) aList\n14   print newList2\n

\n\n

Notice that the expressions being evaluated in lines 11 and 13 are the same. In line 11 we are applying function doubleOddElements to the value of aList and passing this value to the outer function times10Elements. In line 13 we are creating a new function from composing two existing functions: times10Elements . doubleOddElements. The parenthesis in line 13 are required because the . operator has lower precedence than the application of function doubleOddElements so without the parenthesis line 13 would evaluate as times10Elements (doubleOddElements aList) which is not what I intended and would throw an error.

The output is:

haskell_tutorial_cookbook_examples/Pure$ stack ghci\n*Main> :l ChainedCalls.hs \n[1 of 1] Compiling ChainedCalls     ( ChainedCalls.hs, interpreted )\nOk, modules loaded: ChainedCalls.\n*ChainedCalls> main\n[0,2,2,6,4,10,6,14,8]\n[0,20,20,60,40,100]\n[0,20,20,60,40,100]\n

\n\n

Using immutable data takes some getting used to. I am going to digress for a minute to talk about working with Haskell. The steps I take when writing new Haskell code are:

\nBe sure I understand the problem\n
\nHow will data be represented - in Haskell I prefer using built-in types when possible\n
\nDetermine which Haskell standard functions, modules, and 3rd party modules might be useful\n
\nWrite and test the pure Haskell functions I think that I need for the application\n
\nWrite an impure main function that fetches required data, calls the pure functions (which are no longer pure in the sense they are called from impure code), and saves the processed data.\n

I am showing you many tiny examples but please keep in mind the entire process of writing longer programs.

Error Handling

We have seen examples of handling soft errors when no value can be calculated: use Maybe, Just, and Nothing. In bug free pure Haskell code, runtime exceptions should be very rare and I usually do not try to trap them.

Using Maybe, Just, and Nothing is much better than, for example, throwing an error using the standard function error:

1 *Main> error \"test error 123\"\n2 *** Exception: test error 123\n

\n\n

and then, in impure code catching the errors, here is the documentation for your reference.

In impure code that performs IO or accesses network resources that could possibly run out of memory, etc., runtime errors can occur and you could use the same try catch coding style that you have probably used in other programming languages. I admit this is my personal coding style but I don’t like to catch runtime errors. I spent a long time writing Java applications and when possible I preferred using uncaught exceptions and I usually do the same when writing impure Haskell code.

Because of Haskell’s type safety and excellent testing tools, it is possible to write nearly error free Haskell code. Later when we perform network IO we will rely on library support to handle errors and timeouts in a clean “Haskell like” way.

Testing Haskell Code

The example in this section is found in the directory haskell_tutorial_cookbook_examples/TestingHaskell.

If you use stack to create a new project then the framework for testing is generated for you:

 1 $ stack new TestingHaskell\n 2 $ cd TestingHaskell\n 3 $  ls -R\n 4 LICENSE         app         test\n 5 Setup.hs        src\n 6 TestingHaskell.cabal    stack.yaml\n 7 \n 8 TestingHaskell//app:\n 9 Main.hs\n10 \n11 TestingHaskell//src:\n12 Lib.hs\n13 \n14 TestingHaskell//test:\n15 Spec.hs\n16 \n17 $ cat test/Spec.hs \n18 main :: IO ()\n19 main = putStrLn \"Test suite not yet implemented\"\n20 $ stack setup\n21 $ stack build\n

\n\n

This stack generated project is more complex than the project I created manually in the directory haskell_tutorial_cookbook_examples/Pure. The file Setup.hs is a placeholder and uses any module named Main in the app directory. This module, defined in app/Main.hs, imports the module Lib defined in src/Lib.hs.

The generated test does not do anything, but let’s run it anyway:

$ stack test\nRegistering TestingHaskell-0.1.0.0...\nTestingHaskell-0.1.0.0: test (suite: TestingHaskell-test)\n             \nProgress: 1/2 Test suite not yet implemented\n             \nCompleted 2 action(s).\n

\n\n

In the generated project, I made a few changes:

\nremoved src/Lib.hs\n
\nadded src/MyColors.hs providing the type MyColors that we defined earlier\n
\nmodified app/Main.hs to use the MyColors type\n
\nadded tests to test/Spec.hs\n

Here is the contents of TestingHaskell/src/MyColors.hs:

module MyColors where\n\ndata MyColors = Orange | Red | Blue | Green | Silver\n deriving (Show, Eq)\n          \ninstance Ord MyColors where\n  compare c1 c2 = compare (show c1) (show c2)\n

\n\n

And the new test/Spec.hs file:

 1 import Test.Hspec\n 2 \n 3 import MyColors\n 4 \n 5 main :: IO ()\n 6 main = hspec spec\n 7 \n 8 spec :: Spec\n 9 spec = do\n10   describe \"head\" $ do\n11     it \"test removing first list element\" $ do\n12       head [1,2,3,4] `shouldBe` 1\n13       head [\"the\", \"dog\", \"ran\"] `shouldBe` \"dog\" -- should fail\n14   describe \"MyColors tests\" $ do\n15     it \"test custom 'compare' function\" $ do\n16       MyColors.Green < MyColors.Red `shouldBe` True\n17       Red > Silver `shouldBe` True                -- should fail\n

\n\n

Notice how two of the tests are meant to fail as an example. Let’s run the tests:

 1 $ stack test\n 2 TestingHaskell-0.1.0.0: test (suite: TestingHaskell-test)\n 3              \n 4 Progress: 1/2\n 5 head\n 6   test removing first list element FAILED [1]\n 7 MyColors tests\n 8   test custom 'compare' function FAILED [2]\n 9 \n10 Failures:\n11 \n12   test/Spec.hs:13: \n13   1) head test removing first list element\n14        expected: \"dog\"\n15         but got: \"the\"\n16 \n17   test/Spec.hs:17: \n18   2) MyColors tests test custom 'compare' function\n19        expected: True\n20         but got: False\n21 \n22 Randomized with seed 1233887367\n23 \n24 Finished in 0.0139 seconds\n25 2 examples, 2 failures\n26              \n27 Completed 2 action(s).\n28 Test suite failure for package TestingHaskell-0.1.0.0\n29     TestingHaskell-test:  exited with: ExitFailure 1\n30 Logs printed to console\n

\n\n

In line one with stack test we are asking stack to run app tests in the subdirectory test. All Haskell source files in subdirectory test are assumed to be test files. In the listing for file test/Spec.hs we have two tests that fail on purpose and you see the output for the failed tests at lines 12-15 and 17-20.

Because the Haskell compiler does such a good job at finding type errors I have fewer errors in my Haskell code compared to languages like Ruby and Common Lisp. As a result I find myself writing fewer tests for my Haskell code than I would write in other languages. Still, I recommend some tests for each of your projects; decide for yourself how much relative effort you want to put into writing tests.

Pure Haskell Wrap Up

I hope you are starting to get an appreciation for using composition of functions and higher order functions to enable us to compose programs from smaller pieces that can be joined together.

This composition is made easier when using pure functions that always return the same value when called with the same type of arguments.

We will continue to see examples of how lazy evaluation simplifies code because we can use infinitely large lists with the assurance that values are not calculated until they are needed.

In addition to Haskell code generally having fewer errors (after it gets by the compiler!) other advantages of functional programming include more concise code that is easy to read and understand once you get some experience with the language.

Tutorial on Impure Haskell Programming

One of the great things about Haskell is that the language encourages us to think of our code in two parts:

\nPure functional code (functions have no side effects) that is easy to write and test. Functional code tends to be shorter and less likely to be imperative (i.e., more functional, using maps and recursion, and less use of loops as in Java or C++).\n
\nImpure code that deals with side effects like file and network IO, maintaining state in a type safe way, and isolate imperative code that has side effects.\n

In his excellent Functional Programming with Haskell class at eDX Erik Meijer described pure code as being islands in the ocean and the ocean representing impure code. He says that it is a design decision how much of your code is pure (islands) and how much is impure (the ocean). This model of looking at Haskel programs works for me.

My use the word “impure” is common for referring to Haskell code with side effects. Haskell is a purely functional language and side effects like I/O are best handled in a pure functional way using by wrapping pure values in Mondads.

In addition to showing you reusable examples of impure code that you will likely need in your own programs, a major theme of this chapter is handling impure code in a convenient type safe fashion. Any Monad, which wraps a single value, is used to safely manage state. I will introduce you to using Monad types as required for the examples in this chapter. This tutorial style introduction will prepare you for understanding the sample applications later.

Hello IO () Monad

I showed you many examples of pure code in the last chapter but most examples in source files (as opposed to those shown in a GHCi repl) had a bit of impure code in them: the main function like the following that simply writes a string of characters to standard output:

main = do\n  print \"hello world\"\n

\n\n

The type of function main is:

*Main> :t main\nmain :: IO ()\n

\n\n

The IO () monad is an IO value wrapped in a type safe way. Because Haskell is a lazy evaluation language, the value is not evaluated until it is used. Every IO () action returns exactly one value. Think of the word “mono” (or “one”) when you think of Monads because they always return one value. Monads are also used to connnect together parts of a program.

What is it about the function main in the last example that makes its type an IO ()? Consider the simple main function here:

module NoIO where\n\nmain = do\n  let i = 1 in\n    2 * i\n

\n\n

and its type:

*Main> :l NoIO\n[1 of 1] Compiling NoIO             ( NoIO.hs, interpreted )\nOk, modules loaded: NoIO.\n*NoIO> main\n2\n*NoIO> :t main\nmain :: Integer\n*NoIO> \n

\n\n

OK, now you see that there is nothing special about a main function: it gets its type from the type of value returned from the function. It is common to have the return type depend on the function argument types. The first example returns a type IO () because it returns a print do expression:

*Main> :t print\nprint :: Show a => a -> IO ()\n*Main> :t putStrLn\nputStrLn :: String -> IO ()\n

\n\n

The function print shows the enclosing quote characters when displaying a string while putStrLn does not. In the first example, what happens when we stitch together several expressions that have type IO ()? Consider:

main = do\n  print 1\n  print \"cat\"\n

\n\n

Function main is still of type IO (). You have seen do expressions frequently in examples and now we will dig into what the do expression is and why we use it.

The do notation makes working with monads easier. There are alternatives to using do that we will look at later.

One thing to note is that if you are doing bindings inside a do expression using a let with a in expression, you need to wrap the bindings in a new (inner) do expression if there is more than one line of code following the let statement. The way to avoid requiring a nested do expression is to not use in in a let expression inside a do block of code. Yes, this sounds complicated but let’s clear up any confusion by looking at the examples found in the file ImPure/DoLetExample.hs (you might also want to look at the similar example file ImPure/DoLetExample2.hs that uses bind operators instead of a do statement; we will look at bind operators in the next section):

module DoLetExample where\n  \nexample1 = do  -- good style\n  putStrLn \"Enter an integer number:\"\n  s <- getLine\n  let number = (read s :: Int) + 2\n  putStrLn $ \"Number plus 2 = \" ++ (show number)\n\nexample2 = do  -- avoid using \"in\" inside a do statement\n  putStrLn \"Enter an integer number:\"\n  s <- getLine\n  let number = (read s :: Int) + 2 in\n    putStrLn $ \"Number plus 2 = \" ++ (show number)\n\nexample3 = do  -- avoid using \"in\" inside a do statement\n  putStrLn \"Enter an integer number:\"\n  s <- getLine\n  let number = (read s :: Int) + 2 in\n    do -- this do is required since we have two dependent statements:\n      putStrLn \"Result is:\"\n      putStrLn $ \"Number plus 2 = \" ++ (show number)\n\nmain = do\n  example1\n  example2\n  example3\n

\n\n

You should use the pattern in function example1 and not the pattern in example2. The do expression is syntactic sugar that allows programmers to string together a sequence of operations that can mix pure and impure code.

To be clear, the left arrow <- is used when the expression on the right side is some type of IO () that needs to be lifted before being used. A let do expression is used when the right side expression is a pure value.

On lines 6 and 12 we are using function read to converting a string read out of IO String () to an integer value. Remember that the value of s (from calling readLine) is an IO () so in the same way you might read from a file, in this example we are reading a value from an IO () value.

A Note About >> and >>= Operators

So far in this book I have been using the syntactic sugar of the do expression to work with Monads like IO () and I will usually use this syntactic sugar for the rest of this book.

Even though I find it easier to write and read code using do, many Haskell programmers prefer >> and >>= so let’s go over these operators so you won’t be confused when reading Haskell code that uses them. Also, when we use do expressions in code the compiler generates similar code using these >> and >>= operators.

The Monad type class defines the operators >>= and return. We turn to the GHCi repl to experiment with and learn about these operators:

*Main> :t (>>)\n(>>) :: Monad m => m a -> m b -> m b\n*Main> :t (>>=)\n(>>=) :: Monad m => m a -> (a -> m b) -> m b\n*Main> :t return\nreturn :: Monad m => a -> m a\n

\n\n

We start with the return function type return :: Monad m => a -> m a which tells us that for a monad m the function return takes a value and wraps it in a monad. We will see examples of the return function used to return a wrapped value from a function that returns IO () values. The bind operator (>>) is used to evaluate two expressions in sequence. As an example, we can replace this do expression:

main = do\n  example1\n  example2\n  example3\n

\n\n

with the following:

main = example1 >> example2 >> example3\n

\n\n

The operator >>= is similar to >> except that it evaluates the left-hand expression and pipes its value into the right-hand side expression. The left-hand side expression is evaluated to some type of IO () and the expression on the right-hand side typically reads from the input IO (). An example will make this simpler to understand:

module DoLetExample3 where\n  \nexample3 =  putStrLn \"Enter an integer number:\" >>  getLine\n\nexample4 mv = do\n  let number = (read mv :: Int) + 2\n  putStrLn $ \"Number plus 2 = \" ++ (show number)\n\nmain = example3 >>= example4\n

\n\n

Note that I could have used a do statement to define function example3 but used a bind operator instead. Let’s run this example and look at the function types. Please don’t just quickly read through the following listing; when you understand what is happening in this example then for the rest of your life programming in Haskell things will be easier for you:

 1 *DoLetExample3> main\n 2 Enter an integer number:\n 3 1\n 4 Number plus 2 = 3\n 5 *DoLetExample3> :t example3\n 6 example3 :: IO String\n 7 *DoLetExample3> :t example4\n 8 example4 :: String -> IO ()\n 9 *DoLetExample3> :t main\n10 main :: IO ()\n11 *DoLetExample3> let x = example3\n12 *DoLetExample3> x\n13 Enter an integer number:\n14 4\n15 \"4\"\n16 *DoLetExample3> :t x\n17 x :: IO String\n18 *DoLetExample3> x >>= example4\n19 Enter an integer number:\n20 3\n21 Number plus 2 = 5\n

\n\n

The interesting part starts at line 11 when we define x to be the returned value from calling example3. Remember that Haskell is a lazy language: evaluation is postponed until a value is actually used.

Working inside a GHCi repl is like working interactively inside a do expression. When we evaluate x in line 12 then the code in function example3 is actually executed (notice this is where the user prompt to enter a number occurs). In line 18 we are re-evaluationg the value in x and passing the resulting IO String () value to the function example4.

Haskell is a “piecemeal” programming language as are the Lisp family of languages where a repl is used to write little pieces code that are collected into programs. For simple code in Haskell (and Lisp languages) I do sometimes directly enter code into a text editor but very ofter I start in a repl, experiment, debug, refine, and then copy into an edited file.

Console IO Example with Stack Configuration

The directory CommandLineApps contains two simple applications that interact with STDIO, that is to write to the console and read from the keyboard. The first example can be found in file CommandLineApp/CommandLine1.hs:

 1 module Main where\n 2   \n 3 import System.IO\n 4 import Data.Char (toUpper)\n 5 \n 6 main = do\n 7   putStrLn \"Enter a line of text for test 1:\"\n 8   s <- getLine\n 9   putStrLn $ \"As upper case:\\t\" ++ (map toUpper s)\n10   main\n

\n\n

Lines 3 and 4 import the entire System.IO module (that is, import all exported symbols from System.IO) and just the function toUpper from module Data.Char. System.IO is a standard Haskell module and we do not have to do anything special to import it. The Data.Char is stored in the package text. The package text is contained in the library package base which is specifies in the CommandLineApp.cabal configuration file that we will look at soon.

Use of the <- assignment in line 8 in the last Haskell listing is important to understand. It might occur to you to leave out line 8 and just place the getLine function call directly in line 9, like this:

  putStrLn $ \"As upper case:\\t\" ++ (map toUpper getLine)\n

\n\n

If you try this (please do!) you will see compilation errors like:

  Couldn't match expected type ‘[Char]’ with actual type ‘IO String’\n  In the second argument of ‘map’, namely ‘getLine’\n  In the second argument of ‘(++)’, namely ‘(map toUpper getLine)’\n

\n\n

The type of getLine is an IO () that is a wrapped IO call. The value is not computed until it is used. The <- assignment in line 8 evaluates the IO call and unwraps the result of the IO operation so that it can be used.

I don’t spend much time covering stack project configuration files in this book but I do recommend that as you work through examples to also look for a file in each example directory ending with the file extension .cabal that specified which packages need to be loaded. For some examples it might take a while to download and configure libraries the first time you run either stack build or stack ghci in an example directory.

The Haskell stack project in the CommandLineApp directory has five target applications as we can see in the CommandLineApp.cabal file. I am not going to go into much detail about the project cabal and stack.yaml files generated by stack when you create a new project except for configuration data that I had to add manually; in this case, I added two executable targets at the end of the cabal file (note: the project in the github repository for this book has more executable targets, I just show a few here):

 1 executable CommandLine1\n 2   hs-source-dirs:      .\n 3   main-is:             CommandLine1.hs\n 4   default-language:    Haskell2010\n 5   build-depends:       base >= 4.7 && < 5\n 6   \n 7 executable CommandLine2\n 8   hs-source-dirs:      .\n 9   main-is:             CommandLine2.hs\n10   default-language:    Haskell2010\n11   build-depends:       base >= 4.7 && < 5\n12 \n13 executable ReadTextFile\n14   hs-source-dirs:      .\n15   main-is:             ReadTextFile.hs\n16   default-language:    Haskell2010\n17   build-depends:       base >= 4.7 && < 5\n18 \n19 executable GameLoop1\n20   hs-source-dirs:      .\n21   main-is:             GameLoop1.hs\n22   default-language:    Haskell2010\n23   build-depends:       base >= 4.7 && < 5, time\n24         \n25 executable GameLoop2\n26   hs-source-dirs:      .\n27   main-is:             GameLoop2.hs\n28   default-language:    Haskell2010\n29   build-depends:       base >= 4.7 && < 5, random\n

\n\n

The executable name determines the compiled and linked executable file name. For line 1, an executable file “CommandLine1” (or “CommandLine1.exe”“ on Windows) will be generated. The parameter hs-source-dirs is a comma separated list of source file directories. In this simple example all Haskell source files are in the project’s top level directory “../”. The build-depends is a comma separated list of module libraries; here we only use the base built-in modules packaged with Haskell.

Let’s use a GHCi repl to poke at this code and understand it better. The project defined in CommandLineApp/CommandLineApp.cabal contains many executable targets so when we enter a GHCi repl, the available targets are shown and you can choose one; in this case I am selecting the first target defined in the cabal file. In later GHCi repl listings, I will edit out this output for brevity:

 1 $ stack ghci\n 2 \n 3 * * * * * * * *\n 4 The main module to load is ambiguous. Candidates are: \n 5 1. Package `CommandLineApp' component exe:CommandLine1 with main-is file: /Users/mar\\\n 6 kw/GITHUB/haskell_tutorial_cookbook_examples/CommandLineApp/CommandLine1.hs\n 7 2. Package `CommandLineApp' component exe:CommandLine2 with main-is file: /Users/mar\\\n 8 kw/GITHUB/haskell_tutorial_cookbook_examples/CommandLineApp/CommandLine2.hs\n 9 3. Package `CommandLineApp' component exe:ReadTextFile with main-is file: /Users/mar\\\n10 kw/GITHUB/haskell_tutorial_cookbook_examples/CommandLineApp/ReadTextFile.hs\n11 You can specify which one to pick by: \n12  * Specifying targets to stack ghci e.g. stack ghci CommandLineApp:exe:CommandLine1\n13  * Specifying what the main is e.g. stack ghci --main-is CommandLineApp:exe:CommandL\\\n14 ine1\n15  * Choosing from the candidate above [1..3]\n16 * * * * * * * *\n17 \n18 Specify main module to use (press enter to load none): 1\n19 Loading main module from cadidate 1, --main-is /Users/markw/GITHUB/haskell_tutorial_\\\n20 cookbook_examples/CommandLineApp/CommandLine1.hs\n21 \n22 Configuring GHCi with the following packages: CommandLineApp\n23 GHCi, version 7.10.3: http://www.haskell.org/ghc/  :? for help\n24 Ok, modules loaded: none.\n25 [1 of 1] Compiling Main             ( /Users/markw/GITHUB/haskell_tutorial_cookbook_\\\n26 examples/CommandLineApp/CommandLine1.hs, interpreted )\n27 Ok, modules loaded: Main.\n28 *Main> :t main\n29 main :: IO b\n30 *Main> :info main\n31 main :: IO b\n32 -- Defined at /Users/markw/GITHUB/haskell_tutorial_cookbook_examples/CommandLineApp/\\\n33 CommandLine1.hs:6:1\n34 *Main> :t getLine\n35 getLine :: IO String\n36 *Main> :t putStrLn\n37 putStrLn :: String -> IO ()\n38 *Main> main\n39 Enter a line of text for test 1:\n40 line 1\n41 As upper case:  LINE 1\n42 Enter a line of text for test 1:\n43 line 2\n44 As upper case:  LINE 2\n45 Enter a line of text for test 1:\n46 ^C Interrupted.\n47 *Main> \n

\n\n

In line 36 the function getLine is of type getLine :: IO String which means that calling getLine returns a value that is a computation to get a line of text from stdio but the IO operation is not performed until the value is used.

Please note that it is unusual to put five executable targets in a project’s cabal file. I am only doing so here because I wanted to group five similar examples together in this subdirectory of the github repo for this book. This repo has 16 example subdirectories, and the number would be much greater if I didn’t collect similar examples together.

We will use the example in file CommandLine2.hs in the next section which is similar to this example but also appends the user input to a text file.

File IO

We will now look at a short example of doing file IO. We will write Haskell simple string values to a file. If you are using the more efficient Haskell Text values, the code is the same. Text values are more efficient than simple string values when dealing with a lot of data and we will later use a compiler setting to automatically convert between the underlying formats. The following listing shows CommandLineApp/CommandLine2.hs:

 1 module Main where\n 2   \n 3 import System.IO\n 4 import Data.Char (toUpper)\n 5 \n 6 main = do\n 7   putStrLn \"Enter a line of text for test2:\"\n 8   s <- getLine\n 9   putStrLn $ \"As upper case:\\t\" ++ (map toUpper s)\n10   appendFile \"temp.txt\" $ s ++ \"\\n\"\n11   main\n

\n\n

Note the use of recursion in line 11 to make this program loop forever until you use a Control-c to stop the program.

In line 10 we are using function appendFile to open a file, append a string to it, and then close the file. appendFile is of type appendFile :: FilePath -> String -> IO (). It looks like we are passing a simple string as a file name instead of type FilePath but if you look up the definition of FilePath you will see that it is just an alias for string: type FilePath = String.

Running this example in a GHCi repl, with much of the initial printout from running stack ghci not shown:

\n\n

$ stack ghci\nCommandLineApp-0.1.0.0: configure\nSpecify main module to use (press enter to load none): 2\nOk, modules loaded: Main.\n*Main> main\nEnter a line of text for test2:\nline 1\nAs upper case: LINE 1\nEnter a line of text for test2:\nline 2\nAs upper case: LINE 2\nEnter a line of text for test2:\n^C Interrupted.\n*Main>

 1 The file *temp.txt* was just created.\n 2 \n 3 The next example used *ReadTextFile.hs* to read the file *temp.txt* and process the \\\n 4 text by finding all words in the file:\n 5 \n 6 ```haskell{line-numbers: true}\n 7 module Main where\n 8   \n 9 import System.IO\n10 import Control.Monad\n11 \n12 main = do\n13   entireFileAsString <- readFile \"temp.txt\"\n14   print entireFileAsString\n15   let allWords = words entireFileAsString\n16   print allWords\n

\n\n

readFile is a high-level function because it manages for you reading a file and closing the file handle it uses internally. The built in function words splits a string on spaces and returns a list of strings [String] that are printed on line 7:

 1 $ stack ghci\n 2 CommandLineApp-0.1.0.0: build\n 3 Specify main module to use (press enter to load none): 3\n 4 Ok, modules loaded: ReadTextFile.\n 5 *ReadTextFile> main\n 6 \"line 1\\nline 2\\n\"\n 7 [\"line\",\"1\",\"line\",\"2\"]\n 8 *ReadTextFile> \n 9 *ReadTextFile> :t readFile\n10 readFile :: FilePath -> IO String\n11 *ReadTextFile> :type words\n12 words :: String -> [String]\n

\n\n

What if the function readFile encounters an error? That is the subject for the next section.

Error Handling in Impure Code

I know you have been patiently waiting to see how we handle errors in Haskell code. Your wait is over! We will look at several common types of runtime errors and how to deal with them. In the last section we used the function readFile to read the contents of a text file temp.txt. What if temp.txt does not exist? Well, then we get an error like the following when running the example program in ReadTextFile.hs:

*Main> main\n*** Exception: temp.txt: openFile: does not exist (No such file or directory)\n

\n\n

Let’s modify this last example in a new file ReadTextFileErrorHandling.hs that catches a file not found error. The following example is derived from the first example in Michael Snoyman’s article Catching all exceptions. This example does not work inside threads; if you need to catch errors inside a thread then see the second example in Michael’s article.

 1 module Main where\n 2   \n 3 import System.IO\n 4 import Control.Exception\n 5 \n 6 -- catchAny by Michael Snoyman:\n 7 catchAny :: IO a -> (SomeException -> IO a) -> IO a\n 8 catchAny = Control.Exception.catch\n 9 \n10 safeFileReader :: FilePath -> IO String\n11 safeFileReader fPath = do\n12   entireFileAsString <- catchAny (readFile \"temp.txt\") $ \\error -> do\n13     putStrLn $ \"Error: \" ++ show error\n14     return \"\"\n15   return entireFileAsString\n16   \n17 main :: IO ()\n18 main = do\n19   fContents <- safeFileReader \"temp.txt\"\n20   print fContents\n21   print $ words fContents\n

\n\n

I will run this twice: the first time without the file temp.txt present and a second time with temp.txt in the current directory:

 1 *Main> :l ReadTextFileErrorHandling.hs \n 2 [1 of 1] Compiling Main             ( ReadTextFileErrorHandling.hs, interpreted )\n 3 Ok, modules loaded: Main.\n 4 *Main> main\n 5 Error: temp.txt: openFile: does not exist (No such file or directory)\n 6 \"\"\n 7 []\n 8 1\n 9 *Main> main\n10 \"line 1\\nline 2\\n\"\n11 [\"line\",\"1\",\"line\",\"2\"]\n

\n\n

Until you need to handle runtime errors in a multi-threaded Haskell program, following this example should be sufficient. In the next section we look at Network IO.

Network IO

We will experiment with three network IO examples in this book:

\nA simple socket client/server example in this section.\n
\nReading web pages in the chapter “Web Scraping”\n
\nQuerying remote RDF endpoints in the chapter “Linked Data and the Semantic Web”\n

We start by using a high level library, network-simple for both the client and server examples in the next two sub-sections. The client and server examples are in the directory haskell_tutorial_cookbook_examples/ClientServer in the files Client.hs and Server.hs.

Server Using network-simple Library

The Haskell Network and Network.Simple modules use strings represented as Data.ByteString.Char8 data so as seen in line 1 I set the language type OverloadedStrings. The following example in file ClientServer/Server.hs is derived from an example in the network-simple project:

 1 {-# LANGUAGE OverloadedStrings #-}\n 2 \n 3 module Server where\n 4 \n 5 import Control.Monad\n 6 import qualified Data.ByteString.Char8 as B\n 7 import qualified Network.Simple.TCP as T\n 8 \n 9 reverseStringLoop sock = do\n10   mbs <- T.recv sock 4096\n11   case mbs of\n12     Just bs -> T.send sock (B.reverse bs) >> reverseStringLoop sock\n13     Nothing -> return ()\n14 \n15 main :: IO ()\n16 main = T.withSocketsDo $ do -- derived from library example\n17   T.listen \"*\" \"3000\" $ \\(lsock, laddr) -> do\n18     putStrLn $ \"Listening at \" ++ show laddr\n19     forever . T.acceptFork lsock $ \\(sock, addr) -> do\n20       putStrLn $ \"Connection from \" ++ show addr\n21       reverseStringLoop sock\n

\n\n

The server accepts a string, reverses the string, and returns the reversed string to the client.

I am assuming that you have done some network programming and are familiar with sockets, etc. The function reverseStringLoop defined in lines 9-13 accepts a socket as a parameter and returns a value of type MonadIO that wraps a byte-string value. In line 10 we use the T.recv function that takes two arguments: a socket and the maximum number of bytes to received from the client. The case expression reverses the received byte string, sends the reversed string back to the client, and recursively calls itself waiting for new data from the client. If the client breaks the socket connection, then the function returns an empty MonadIO().

The main function defined in lines 15-21 listens on port 3000 for new client socket connections. In line 19, the function T.acceptFork accepts as an argument a socket value and a function to execute; the complete type is:

1 *Main> :t T.acceptFork\n2 T.acceptFork\n3   :: transformers-0.4.2.0:Control.Monad.IO.Class.MonadIO m =>\n4      T.Socket\n5      -> ((T.Socket, T.SockAddr) -> IO ()) -> m GHC.Conc.Sync.ThreadId\n

\n\n

Don’t let line 3 scare you; the GHCi repl is just showing you where this type of MonadIO is defined. The return type refers to a thread ID that is passed to the function forever :: Monad m => m a -> m b that is defined in the module Control.Monad and lets the thread run until it terminates.

The network-simple package is fairly high level and relatively simple to use. If you are interested you can find many client/server examples on the web that use the lower-level network package.

We will develop a client application to talk with this server in the next section but if you want to immediately try the server, start it and then run telnet in another terminal window:

Prelude> :l Server\n[1 of 1] Compiling Server           ( Server.hs, interpreted )\nOk, modules loaded: Server.\n*Main> main\nListening at 0.0.0.0:3000\n

\n\n

And run telnet:

$ telnet localhost 3000\nTrying 127.0.0.1...\nConnected to localhost.\nEscape character is '^]'.\n12345\n54321\nThe dog ran down the street\nteerts eht nwod nar god ehT\n

\n\n

In the next section we write a simple client to talk with this service example.

Client Using network-simple Library

I want to use automatic conversion between strings represented as Data.ByteString.Char8 data and regular [Char] strings so as seen in line 1 I set the language type OverloadedStrings in the example in file Client.hs:

 1 {-# LANGUAGE OverloadedStrings #-}\n 2 \n 3 module Client where\n 4 \n 5 import Control.Monad\n 6 import qualified Network.Simple.TCP as T\n 7 \n 8 main = do\n 9   T.connect \"127.0.0.1\" \"3000\" $ \\(connectionSocket, remoteAddr) -> do\n10   putStrLn $ \"Connection established to \" ++ show remoteAddr\n11   T.send connectionSocket \"test123\"\n12   response <- T.recv connectionSocket 100\n13   case response of\n14     Just s -> putStrLn $ \"Response: \" ++ show s\n15     Nothing -> putStrLn \"No response from server\"\n

\n\n

The function T.connect in line 9 accepts arguments for a host name, a port, and a function to call with the connection socket to the server and the server’s address. The body of this inline function, defined in in the middle on line 9 and continuing in lines 10-15, prints the server address, sends a string “test123” to the server, and waits for a response back from the server (T.recv in line 12). The server response is printed, or a warning that no response was received.

While the example in file Server.hs is running in another terminal, we can run the client interactively:

Prelude> :l Client.hs \n[1 of 1] Compiling Client           ( Client.hs, interpreted )\nOk, modules loaded: Client.\n*Main main\nConnection established to 127.0.0.1:3000\nResponse: \"321tset\"\n

\n\n

A Haskell Game Loop that Maintains State Functionally

The example in this section can be found in the file GameLoop2.hs in the directory haskell_tutorial_cookbook_examples/CommandLineApp. This example uses the random package to generate a seed random number for a simple number guessing game. An alternative implementation in GameLoop1.hs, which I won’t discuss, uses the system time to generate a seed.

This is an important example because it demonstrates one way to maintain state in a functional way. We have a read-only game state value that is passed to the function gameLoop which modifies the read-only game state passed as an argument and returns a newly constructed game state as the function’s returned value. This is a common pattern that we will see again later when we develop an application to play a simplified version of the card game Blackjack in the chapter “Haskell Program to Play the Blackjack Card Game.”

 1 module GameLoop2 where\n 2 \n 3 import System.Random\n 4 \n 5 data GameState = GameState { numberToGuess::Integer, numTries::Integer}\n 6                    deriving (Show)\n 7 \n 8 gameLoop :: GameState -> IO GameState\n 9 gameLoop gs = do      \n10   print $ numberToGuess gs\n11   putStrLn \"Enter a number:\"\n12   s <- getLine\n13   let num = read s :: Integer\n14   if num == numberToGuess gs then\n15     return gs\n16   else gameLoop $ GameState (numberToGuess gs) ((numTries gs) + 1)\n17          \n18 main = do\n19   pTime <- randomRIO(1,4)\n20   let gameState = GameState pTime 1\n21   print \"Guess a number between 1 and 4\"\n22   gameLoop gameState\n

\n\n

You notice in line 12 that since we are inside of a do expression we can lift (or unwrap) the IO String () value returned from getLine to a string value that we can use directly. This is a pattern we will use repeatedly. The value returned from getLine is not used until line 13 when we use function read to extract the value from the IO String () value getLine returned.

In the if expression in lines 14-16 we check if the user has input the correct value and can then simply return the input game state to the calling main function. If the user has not guessed the correct number then in line 16 we create a new game state value and call the function gameLoop recursively with the newly constructed game state.

The following listing shows a sample session playing the number guessing game.

Prelude> :l GameLoop2.hs \n[1 of 1] Compiling GameLoop2        ( GameLoop2.hs, interpreted )\nOk, modules loaded: GameLoop2.\n*GameLoop2> main\n\"Guess a number between 1 and 4\"\nEnter a number:\n1\nEnter a number:\n3\nEnter a number:\n4\nGameState {numberToGuess = 4, numTries = 3}\n*GameLoop2> main\n\"Guess a number between 1 and 4\"\nEnter a number:\n1\nEnter a number:\n2\nGameState {numberToGuess = 2, numTries = 2}\n*GameLoop2> \n

\n\n

We will use this pattern for maintaining state in a game in the later chapter “Haskell Program to Play the Blackjack Card Game.”

Efficiency of Haskell Strings

Except for the Client/Server example, so far we have been mostly using simple String values where String is a list of characters [Char]. For longer strings it is much more efficient to use the module Data.Text that is defined in package text (so text needs to be added to the dependencies in your cabal file).

Many Haskell libraries use the simple String type but the use of Data.Text is also common, especially in applications handling large amounts of string data. We have already seen examples of this in the client/server example programs. Fortunately Haskell is a strongly typed language that supports a language extension for automatically handling both simple strings and the more efficient text types. This language extension, as we have seen in a previous example, is activated by adding the following near the top of a Haskell source file:

{-# LANGUAGE OverloadedStrings     #-}\n

\n\n

As much as possible I am going to use simple strings in this book and when we need both simple strings and byte strings I will then use OverloadedStrings for automatic conversion. This conversion is performed by knowing the type signatures of data and functions in surrounding code. The compiler figures out what type of string is expected and does the conversion for you.

A More Detailed Look at Monads

We have been casually using different types of IO () monads. In this section I will introduce you to the State monad and then we will take a deeper look at IO (). While we will be just skimming the surface of the topic of monads, my goal in this section is to teach you enough to work through the remaining examples in this book.

Monads are types belonging to the Monad type class that specifies one operator and one function:

class Monad m where\n  (>>=) :: m a -> (a -> m b) -> m b\n  return :: a -> m a\n

\n\n

The >>= operator takes two arguments: a monad wrapping a value (type a in the above listing) and a function taking the same type a and returning a monad wrapping a new type b. The return value of >>= is a new monad wrapping a value of type b.

The Monad type class function return takes any value and wraps it in a new monad. The naming of return is confusing because it does not alter the flow of execution in a program like a return statement in Java, rather, it wraps a value in a monad.

State Monad

The definition for the constructor of a State monad is:

newtype State s a = State { runState :: s -> (a, s) }\n

\n\n

So far we have been using data to define new types and newtype is similar except newtype acts during compile time and no type information is present at runtime. All monads contain a value and for the State monad this value is a function. The >>= operator is called the bind operator.

The accessor function runState provides the means to access the value in the state. The following example is in the file StateMonad/State1.hs. In this example, incrementState is a state monad that increases its wrapped integer value by one when it is executed. Remember that the return function is perhaps poorly named because it does not immediately “return” from a computation block as it does in other languages; return simply wraps a value as a monad without redirecting the execution flow.

In order to make the following example more clear, I implement the increment state function twice, once using the do notation that you are already familiar with and once using the >>= bind operator:

 1 module Main where\n 2 \n 3 import Control.Monad.State\n 4 \n 5 incrementState :: State Int Int\n 6 incrementState = do\n 7   n <- get\n 8   put (n + 1)\n 9   return n\n10 \n11 -- same state monad without using a 'do' expression:\n12 incrementState2 :: State Int Int\n13 incrementState2 = get >>= \\a ->\n14                   put (a + 1) >>= \\b ->\n15                   return a\n16 \n17 bumpVals (a,b) = (a+1, b+2)\n18 \n19 main = do\n20   print $ runState incrementState 1  -- (1,2) == (return value, final state)\n21   print $ runState incrementState2 1 -- (1,2) == (return value, final state)\n22   print $ runState (mapState bumpVals incrementState) 1 -- (2,4)\n23   print $ evalState incrementState 1  -- 1 == return value\n24   print $ execState incrementState 1  -- 2 == final state\n

\n\n

Here we have used two very different looking, yet equivalent, styles for accessing and modifying state monad values. In lines 6-9 we are using the do notation. The function get in line 7 returns one value: the value wrapped in a state monad. Function put in line 8 replaces the wrapped value in the state monad, in this example by incrementing its numeric value. Finally return wraps the value in a monad.

I am using the runState function defined in lines 20-24 that returns a tuple: the first tuple value is the result of the computation performed by the function passed to runState (incrementState and incrementState2 in these examples) and the second tuple value is the final wrapped state.

In lines 12-15 I reimplemented increment state using the bind function (>>=). We have seen before that >>= passes the value on its left side to the computation on its right side, that is function calls in lines 13-15:

  \\a -> put (a + 1)\n  \\b -> return a\n

\n\n

It is a matter of personal taste whether to code using bind or do. I almost always use the do notation in my own code but I wanted to cover bind both in case you prefer that notation and so you can also read and understand Haskell code using bind. We continue looking at alternatives to the do notation in the next section.

Using Applicative Operators <$> and <*>: Finding Common Words in Files

My goal in this book is to show you a minimal subset of Haskell that is relatively easy to understand and use for coding. However, a big part of using a language is reading other people’s code so I do need to introduce a few more constructs that are widely used: applicative operators.

Before we begin I need to introduce you to a new term: Functor which is a type class that defines only one method fmap. fmap is used to map a function over an IO action and has the type signature:

  fmap :: Functor f => (a -> b) -> f a -> f b\n

\n\n

fmap can be used to apply a pure function like (a -> b) to an IO a and return a new IO b without unwrapping the original IO (). The following short example (in file ImPure/FmapExample.hs) will let you play with this idea:

 1 module FmapExample where\n 2 \n 3 fileToWords fileName = do\n 4   fileText <- readFile fileName\n 5   return $ words fileText\n 6     \n 7 main = do\n 8   words1 <- fileToWords \"text1.txt\"\n 9   print $ reverse words1\n10   words2 <- fmap reverse $ fileToWords \"text1.txt\"\n11   print words2\n

\n\n

In lines 8-9 I am unwrapping the result of the IO [String] returned by the function fileToWords and then applying the pure function words to the unwrapped value. Wouldn’t it be nice to operate on the words in the file without unwrapping the [String] value? You can do this using fmap as seen in lines 10-11. Please take a moment to understand what line 10 is doing. Here is line 10:

  words2 <- fmap reverse $ fileToWords \"text1.txt\"\n

\n\n

First we read the words in a file into an IO [String] monad:

                           fileToWords \"text1.txt\"\n

\n\n

Then we apply the pure function reverse to the values inside the IO [String] monad, creating a new copy:

            fmap reverse $ fileToWords \"text1.txt\"\n

\n\n

Note that from the type of the fmap function, the input monad and output monad can wrap different types. For example, if we applied the function head to an IO [String] we would get an output of IO [Char].

Finally we unwrap the [String] value inside the monad and set words2 to this unwrapped value:

  words2 <- fmap reverse $ fileToWords \"text1.txt\"\n

\n\n

In summary, the Functor type class defines one method fmap that is useful for operating on data wrapped inside a monad.

We will now implement a small application that finds common words in two text files, implementing the primary function three times, using:

\nThe do notation.\n
\nThe >>= bind operator.\n
\nThe Applicative operators <$> and <*>\n

Let’s look at the types for these operators:

(<$>) :: Functor f => (a -> b) -> f a -> f b\n(<*>) :: Applicative f => f (a -> b) -> f a -> f b\n

\n\n

We will use both <$> and <*> in the function commonWords3 in this example and I will explain how these operators work after the following program listing.

This practical example will give you a chance to experiment more with Haskell (you do have a GHCi repl open now, right?). The source file for this example is in the file ImPure/CommonWords.hs:

 1 module CommonWords where\n 2 \n 3 import Data.Set (fromList, toList, intersection)\n 4 import Data.Char (toLower)\n 5 \n 6 fileToWords fileName = do\n 7   fileText <- readFile fileName\n 8   return $ (fromList . words) (map toLower fileText)\n 9   \n10 commonWords file1 file2 = do  \n11   words1 <- fileToWords file1\n12   words2 <- fileToWords file2\n13   return $  toList $ intersection words1 words2\n14 \n15 commonWords2 file1 file2 =\n16   fileToWords file1 >>= \\f1 ->\n17   fileToWords file2 >>= \\f2 ->\n18   return $  toList $ intersection f1 f2\n19                                                             \n20 commonWords3 file1 file2 =\n21   (\\f1 f2 -> toList $ intersection f1 f2)\n22     <$> fileToWords file1\n23     <*> fileToWords file2\n24     \n25 main = do\n26   cw <- commonWords \"text1.txt\" \"text2.txt\"\n27   print cw\n28   cw2 <- commonWords2 \"text1.txt\" \"text2.txt\"\n29   print cw2\n30   cw3 <- commonWords3 \"text1.txt\" \"text2.txt\"\n31   print cw3\n

\n\n

The function fileToWords defined in lines 6-8 simply reads a file, as in the last example, maps contents of the file to lower case, uses words to convert a String to a [String] list of individual words, and uses the function Data.Set.fromList to create a set from a list of words that in general will have duplicates. We are retuning an IO (Data.Set.Base.Set String) value so we can later perform a set intersection operation. In other applications you might want to apply Data.Set.toList before returning the value from fileToWords so the return type of the function would be IO [String].

The last listing defines three similar functions commonWords, commonWords2, and commonWords3.

commonWords defined in lines 10-13 should hopefully look routine and familiar to you now. We set the local variables with the unwrapped (i.e., extracted from a monad) contents of the unique words in two files, and then return monad wrapping the intersection of the words in both files.

The function commonWords2 is really the same as commonWords except that it uses the bind >>= operator instead of the do notation.

The interesting function in this example is commonWords3 in lines 20-23 which uses the applicative operators <$> and <*>. Notice the pure function defined inline in line 21: it takes two arguments of type set and returns the set intersection of the arguments. The operator <$> takes a function on the left side and a monad on the right side which contains the wrapped value to be passed as the argument f1. <*> supplies the value for the inline function arguments f2. To rephrase how lines 21-23 work: we are calling fileToWords twice, both times getting a monad. These two wrapped monad values are passed as arguments to the inline function in line 21 and the result of evaluating this inline function is returned as the value of the function commonWords3.

I hope that this example has at least provided you with “reading knowledge” of the Applicative operators <$> and <*> and has also given you one more example of replacing the do notation with the use of the bind >>= operator.

List Comprehensions Using the do Notation

We saw examples of list comprehensions in the last chapter on pure Haskell programming. We can use return to get lists values that are instances of type Monad:

*Prelude> :t (return [])\n(return []) :: Monad m => m [t]\n*Prelude> :t (return [1,2,3])\n(return [1,2,3]) :: (Monad m, Num t) => m [t]\n*Prelude> :t (return [\"the\",\"tree\"])\n(return [\"the\",\"tree\"]) :: Monad m => m [[Char]]\n

\n\n

We can get list comprehension behavior from the do notation (here I am using the GHCi repl :{ and :} commands to enter multiple line examples):

1 *Main> :{\n2 *Main| do num <- [1..3]\n3 *Main|    animal <- [\"parrot\", \"ant\", \"dolphin\"]\n4 *Main|    return (num, animal)\n5 *Main| :}\n6 [(1,\"parrot\"),(1,\"ant\"),(1,\"dolphin\"),\n7  (2,\"parrot\"),(2,\"ant\"),(2,\"dolphin\"),\n8  (3,\"parrot\"),(3,\"ant\"),(3,\"dolphin\")]\n

\n\n

I won’t use this notation further but you now will recognize this pattern if you read it in other people’s code.

Dealing With Time

In the example in this section we will see how to time a block of code (using two different methods) and how to set a timeout for code that runs in an IO ().

The first way we time a block of code uses getPOSIXTime and can be used to time pure or impure code. The second method using timeIt takes an IO () as an argument; in the following example I wrapped pure code in a print function call which returns an IO () as its value. The last example in the file TimerTest.hs shows how to run impure code wrapped in a timeout.

 1 module Main where\n 2 \n 3 import Data.Time.Clock.POSIX -- for getPOSIXTime\n 4 import System.TimeIt         -- for timeIt\n 5 import System.Timeout        -- for timeout\n 6 \n 7 anyCalculationWillDo n =  -- a function that can take a while to run\n 8   take n $ sieve [2..]\n 9             where\n10               sieve (x:xs) =\n11                 x:sieve [y | y <- xs, rem y x > 0]\n12                 \n13 main = do\n14   startingTime <- getPOSIXTime\n15   print startingTime\n16   print $ last $ take 20000001 [0..]\n17   endingTime <- getPOSIXTime\n18   print endingTime\n19   print (endingTime - startingTime)\n20   timeIt $ print $ last $ anyCalculationWillDo 2000\n21 \n22   let somePrimes = anyCalculationWillDo 3333 in\n23     timeIt $ print $ last somePrimes\n24 \n25   -- 100000 microseconds timeout tests:\n26   timeout 100000 $ print \"simple print **do** expression did not timeout\"\n27   timeout 100000 $ print $ last $ anyCalculationWillDo 4\n28   timeout 100000 $ print $ last $ anyCalculationWillDo 40\n29   timeout 100000 $ print $ last $ anyCalculationWillDo 400\n30   timeout 100000 $ print $ last $ anyCalculationWillDo 4000\n31   timeout 100000 $ print $ last $ anyCalculationWillDo 40000\n32   print $ anyCalculationWillDo 5\n

\n\n

I wanted a function that takes a while to run so for anyCalculationWillDo (lines 7 to 11) I implemented an inefficient prime number generator.

When running this example on my laptop, the last two timeout calls (lines 26 and 31) are terminated for taking more than 100000 microseconds to execute.

The last line 32 of code prints out the first 5 prime numbers greater than 1 so you can see the results of calling the time wasting test function anyCalculationWillDo.

$ stack build --exec TimerTest\n1473610528.2177s\n20000000\n1473610530.218574s\n2.000874s\n17389\nCPU time:   0.14s\n30911\nCPU time:   0.25s\n\"simple print **do** expression did not timeout\"\n7\n173\n2741\n[2,3,5,7,11]\n

\n\n

The timeout function is useful for setting a maximum time that you are willing to wait for a calculation to complete. I mostly use timeout for timing out operations fetching data from the web.

Using Debug.Trace

Inside an IO you can use print statements to understand what is going on in your code when debugging. You can not use print statements inside pure code but the Haskell base library contains the trace functions that internally perform impure writes to stdout. You do not want to use these debug tools in production code.

As an example, I have rewritten the example from the last section to use Debug.Trace.trace and Debug.Trace.traceShow:

 1 module Main where\n 2 \n 3 import Debug.Trace  (trace, traceShow) -- for debugging only!\n 4 \n 5 anyCalculationWillDo n =\n 6   trace\n 7       (\"+++ anyCalculationWillDo: \" ++ show n) $\n 8       anyCalculationWillDo' n\n 9 \n10 anyCalculationWillDo' n =\n11   take n $ trace (\"   -- sieve n:\" ++ (show n)) $ sieve [2..]\n12             where\n13               sieve (x:xs) =\n14                   traceShow (\"     -- inside sieve recursion\") $\n15                             x:sieve [y | y <- xs, rem y x > 0]\n16                 \n17 main = do\n18   print $ anyCalculationWillDo 5\n

\n\n

In line 3 we import the trace and showTrace functions:

*Main> :info trace\ntrace :: String -> a -> a   -- Defined in ‘Debug.Trace’\n*Main> :info traceShow\ntraceShow :: Show a => a -> b -> b  -- Defined in ‘Debug.Trace’\n

\n\n

trace takes two arguments: the first is a string that that is written to stdout and the second is a function call to be evaluated. traceShow is like *trace except that the first argument is converted to a string. The output from running this example is:

+++ anyCalculationWillDo: 5\n   -- sieve n:5\n\"     -- inside sieve recursion\"\n\"     -- inside sieve recursion\"\n\"     -- inside sieve recursion\"\n\"     -- inside sieve recursion\"\n\"     -- inside sieve recursion\"\n[2,3,5,7,11]\n

\n\n

I don’t usually like using the trace functions because debugging with them involves slightly rewriting my code. My preference is to get low level code written interactively in the GHCI repl so it does not need to be debugged. I very frequently use print statement inside IOs since adding them requires no significant modification of my code.

Wrap Up

I tried to give you a general fast-start in this chapter for using monads and in general writing impure Haskell code. This chapter should be sufficient for you to be able to understand and experiment with the examples in the rest of this book.

This is the end of the first section. We will now look at a variety of application examples using the Haskell language.

While I expect you to have worked through the previous chapters in order, for the rest of the book you can skip around and read the material in any order that you wish.

Section 2 - Cookbook

Now that you have worked through the pure and impure Haskell coding tutorials in the first two chapters we will look at a “cookbook” of techniques and sample applications to solve some common programming tasks as well as implement a program to play the card game Blackjack.

I expect you, dear reader, to have studied and absorbed the tutorial material on pure and impure Haskell programming in the first two chapters. If you are new to Haskell, or don’t have much experience yet, carefully working through these tutorial chapters is a requirement for understanding the material in the rest of this book.

This section contains the following “recipe” applications:

\nUsing the OpenAI APIs for text completion\n
\nWeb app for calling Google Gemini AIs\n
\nCommand line application for calling Google Gemini AIs\n
\nCommand line application for calling Ollama LLM server (for local models)\n
\nUsing the Brave Search API\n
\nTextprocessing CSV Files\n
\nTextprocessing JSON Files\n
\nUsing sqlite and Postgres databases\n
\nREST Server Providing JSON Data\n
\nREST Client\n
\nAccessing and Using Linked Data\n
\nQuerying Semantic Web RDF Data Sources\n
\nAnnotating English text with Wikipedia/DBPedia URIs for entities in the original text. Entities can be people, places, organizations, etc.\n
\nPlay the Blackjack card game\n

Using the OpenAI Large Language Model APIs in Haskell

Here we will use the library openai-hs written by Alexander Thiemann. The GitHub repository for his library is https://github.com/agrafix/openai-hs/tree/main/openai-hs.

We will start by writing client code to call the OpenAI completion API for an arbitrary input text. We will then use this completion client code for a specialized application: finding all the place names in input text and returning them as a list of strings.

In the development of practical AI systems, LLMs like those provided by OpenAI, Anthropic, and Hugging Face have emerged as pivotal tools for numerous applications including natural language processing, generation, and understanding. These models, powered by deep learning architectures, encapsulate a wealth of knowledge and computational capabilities. As a Haskell enthusiast embarking on the journey of intertwining the elegance of Haskell with the power of these modern language models, you might also want to experiment with the OpenAI Python examples that are much more complete than what we look at here.

OpenAI provides an API for developers to access models like GPT-4o. The OpenAI API is designed with simplicity and ease of use in mind, making it a common choice for developers. It provides endpoints for different types of interactions, be it text completion, translation, or semantic search among others. We will use the text completion API in this chapter. The robustness and versatility of the OpenAI API make it a valuable asset for anyone looking to integrate advanced language understanding and generation capabilities into their applications.

While we use the GPT-4o model here, you can substitute the following models:

\nGPT-4o - expensive to run, for complex multi-step tasks\n
\nGPT-4o-mini - inexpensive to run, for simpler tasks (this is the default model I use)\n
\no1-preview - very expensive to run, most capable model that has massive knowledge of the real world and can solve complex multi-step reasoning problems.\n
\no1-mini - slightly less expensive to run than o1-preview, less real world knowledge and simpler reasoning capabilities.\n

Example Client Code

This Haskell program demonstrates how to interact with the OpenAI ChatCompletion API using the Openai-hs library. The code sends a prompt to the OpenAI API and prints the assistant’s response to the console. It’s a practical example of how to set up an OpenAI client, create a request, handle the response, and manage potential errors in a Haskell application.

Firstly, the code imports necessary modules and libraries. It imports OpenAI.Client for interacting with the OpenAI API and Network.HTTP.Client along with Network.HTTP.Client.TLS for handling HTTP requests over TLS. The System.Environment module is used to access environment variables, specifically to retrieve the OpenAI API key. Additionally, Data.Text is imported for efficient text manipulation, and Data.Maybe is used for handling optional values.

The core of the program is the completionRequestToString function. This function takes a String argument prompt and returns an IO String, representing the assistant’s response.

What is an IO String? In Haskell, IO String represents an action that, when executed, produces a String, whereas String is simply a value.

\nIO String: A computation in the IO monad that will produce a String when executed.\n
\nString: A pure value, just a sequence of characters.\n

You can’t directly extract a String from IO String; you need to perform the IO action (e.g., using main or inside the do notation) to get the result.

Inside function completionRequestToString, an HTTP manager with TLS support is created using newManager tlsManagerSettings. Then, it retrieves the OpenAI API key from the OPENAI_KEY environment variable using getEnv “OPENAI_KEY” and packs it into a Text type with T.pack.

An OpenAI client is instantiated using makeOpenAIClient, passing the API key, the HTTP manager, and an integer 4, which represents a maximum number of retries. The code then constructs a ChatCompletionRequest, specifying the model to use (in this case, ModelId “gpt-4o”) and the messages to send. The messages consist of a single ChatMessage with the user’s prompt, setting chmContent to Just (T.pack prompt) and chmRole to “user”. All other optional parameters in the request are left as Nothing, implying default values will be used.

The function then sends the chat completion request using completeChat client request and pattern matches on the result to handle both success and failure cases. If the request fails (Left failure), it returns a string representation of the failure. On success (Right success), it extracts the assistant’s reply from the chrChoices field. It unpacks the content from Text to String, handling the case where content might be absent by providing a default message “No content”.

Finally, the function main serves as the entry point of the program. It calls completionRequestToString with the prompt “Write a hello world program in Haskell” and prints the assistant’s response using putStrLn. This demonstrates how to use the function in a real-world scenario, providing a complete example of sending a prompt to the OpenAI API and displaying the result.

 1 {-# LANGUAGE OverloadedStrings #-}\n 2 import OpenAI.Client\n 3 \n 4 import Network.HTTP.Client\n 5 import Network.HTTP.Client.TLS\n 6 import System.Environment (getEnv)\n 7 import qualified Data.Text as T\n 8 import Data.Maybe (fromMaybe)\n 9 import Data.Text (splitOn)\n10 \n11 -- example derived from the openai-client library documentation\n12 \n13 completionRequestToString :: String -> IO String\n14 completionRequestToString prompt = do\n15     manager <- newManager tlsManagerSettings\n16     apiKey <- T.pack <$> getEnv \"OPENAI_KEY\"\n17     let client = makeOpenAIClient apiKey manager 4\n18     let request = ChatCompletionRequest\n19                  { chcrModel = ModelId \"gpt-4o\"\n20                  , chcrMessages =\n21                     [ ChatMessage\n22                         { chmContent = Just (T.pack prompt)\n23                         , chmRole = \"user\"\n24                         , chmFunctionCall = Nothing\n25                         , chmName = Nothing\n26                         }\n27                     ]\n28                  , chcrFunctions = Nothing\n29                  , chcrTemperature = Nothing\n30                  , chcrTopP = Nothing\n31                  , chcrN = Nothing\n32                  , chcrStream = Nothing\n33                  , chcrStop = Nothing\n34                  , chcrMaxTokens = Nothing\n35                  , chcrPresencePenalty = Nothing\n36                  , chcrFrequencyPenalty = Nothing\n37                  , chcrLogitBias = Nothing\n38                  , chcrUser = Nothing\n39                  }\n40     result <- completeChat client request\n41     case result of\n42         Left failure -> return (show failure)\n43         Right success ->\n44             case chrChoices success of\n45                 (ChatChoice {chchMessage = ChatMessage {chmContent = content}} : _) \\\n46 ->\n47                     return $ fromMaybe \"No content\" $ T.unpack <$> content\n48                 _ -> return \"No choices returned\"\n49 \n50 main :: IO ()\n51 main = do\n52     response <- completionRequestToString \"Write a hello world program in Haskell\"\n53     putStrLn response\n

\n\n

Here is sample output generated by the gpt-4o OpenAI model:

 1 $ cabal build\n 2 Build profile: -w ghc-9.8.1 -O1\n 3 $ cabal run  \n 4 Sure! Here is a simple \"Hello, World!\" program in Haskell:\n 5 \n 6   main :: IO ()\n 7   main = putStrLn \"Hello, World!\"\n 8 \n 9 Explanation:\n10 - `main :: IO ()` declares that `main` is an I/O action that returns no meaningful v\\\n11 alue (indicated by `()`).\n12 - `putStrLn \"Hello, World!\"` is the I/O action that outputs the string \"Hello, World\\\n13 !\" followed by a newline.\n14 \n15 To run this program:\n16 1. Save the code in a file, for example, `hello.hs`.\n17 2. Open a terminal.\n18 3. Navigate to the directory containing `hello.hs`.\n19 4. Run the program using the Haskell compiler/interpreter (GHC). You can do this by \\\n20 running:\n21 \n22    runhaskell hello.hs\n23 \n24 or by compiling it and then running the executable:\n25 \n26    ghc -o hello hello.hs\n27    ./hello\n

\n\n

For completeness, here is a partial listing of the OpenAiApiClient.cabal file:

 1 name:                OpenAiApiClient\n 2 version:             0.1.0.0\n 3 author:              Mark watson and the author of OpenAI Client library Alexander T\\\n 4 hiemann\n 5 build-type:          Simple\n 6 cabal-version:       >=1.10\n 7 \n 8 executable GenText\n 9   hs-source-dirs:      .\n10   main-is:             GenText.hs\n11   default-language:    Haskell2010\n12   build-depends:       base >= 4.7 && < 5, mtl >= 2.2.2, text, http-client >= 0.7.13\\\n13 .1, openai-hs, http-client-tls\n

\n\n

Adding a Simple Application: Find Place Names in Input Text

The example file GenText.hs contains a small application example that uses the function completionRequestToString prompt that we defined in the last section.

Here we define a new function:

1 findPlaces :: String -> IO [String]\n2 findPlaces text = do\n3     let prompt = \"Extract only the place names separated by commas from the followin\\\n4 g text:\\n\\n\" ++ text\n5     response <- completionRequestToString prompt \n6     -- Convert Text to String using T.unpack before filtering\n7     let places = filter (not . null) $ map T.unpack $ splitOn \",\" (T.pack response) \n8     -- Strip leading and trailing whitespace from each place name\n9     return $ map (T.unpack . T.strip . T.pack) places\n

\n\n

The function findPlaces extracts a list of place names from a given text using an LLM (Large Language Model).

\nIt constructs a prompt instructing the LLM to extract only comma-separated place names.\n
\nIt sends this prompt to the LLM using the completionRequestToString function.\n
\nIt processes the LLM’s response, splitting it into a list of potential place names, filtering out empty entries, and stripping leading/trailing whitespace.\n
\nIt returns the final list of extracted place names.\n

You should use the function findPlaces as a template for prompting the OpenAI completion models like GPT-4o to perform specific tasks.

Given the example code:

1 main :: IO ()\n2 main = do\n3     places <- findPlaces \"I visited London, Paris, and New York last year.\"\n4     print places \n

\n\n

The output would look like:

1 [\"London\",\"Paris\",\"New York\"]\n

\n\n

A Web Application For Using the Google Gemini APIs

Here we look at a web application using the Google Gemini API for chat.\nWe’ll show the complete program listing that demonstrates how to interact with Gemini, providing a foundation for building more complex applications. This example code encapsulates the core concepts of sending requests and processing responses, showcasing Haskell’s ability to integrate with modern APIs while maintaining code clarity and robustness.

This example is in the directory haskell_tutorial_cookbook_examples/webchat.

Our focus will be on understanding the data structures used to represent Gemini requests and responses, as well as the HTTP client functionality employed to communicate with the API. We’ll explore how Haskell’s type system aids in constructing well-defined requests and how its data handling capabilities simplify the parsing of complex JSON responses. Through this exploration, you’ll gain valuable insights into leveraging Haskell’s strengths for real-world API interactions.

Beyond the immediate application of the Gemini API, this chapter serves as a template for interacting with web services in Haskell. The techniques and patterns discussed here can be readily adapted to other APIs and use cases. By the end of this chapter, you’ll be equipped with the knowledge and tools to confidently integrate Haskell into your projects that involve communication with external services, opening up a world of possibilities for building robust and scalable applications.

Here is an example of the web app:

\n\n

NOTE: I had to add “debug printout” to this example to get the API calls working correctly. I decided to leave the debug printout in the code so when you run the web app you can see what the request and response data looks like.

You need a Google API key to run this example. Set the environment variable GOOGLE_API_KEY to your key.

This Haskell code defines data types for interacting with the Google Gemini API and sets up a simple web server using the Scotty framework:

import Web.Scotty\nimport Network.HTTP.Client.TLS (tlsManagerSettings)\nimport Network.HTTP.Client\nimport Network.HTTP.Types.Status (status500)\nimport qualified Data.ByteString.Char8 as BS\nimport qualified Data.Text.Lazy as TL\nimport qualified Data.Aeson as Aeson\nimport Data.Aeson (FromJSON, ToJSON)\nimport GHC.Generics\nimport qualified Data.Text as T\nimport System.IO (hFlush, stdout)\nimport System.Environment (getEnv)\nimport qualified Data.Vector as V\n\ndata GeminiRequest = GeminiRequest\n  { prompt :: String\n  } deriving (Show, Generic, FromJSON, ToJSON)\n\ndata GeminiResponse = GeminiResponse\n  { candidates :: [Candidate]\n  , promptFeedback :: Maybe PromptFeedback\n  } deriving (Show, Generic, FromJSON, ToJSON)\n\ndata Candidate = Candidate\n  { content :: Content2\n  , finishReason :: Maybe String\n  , index :: Maybe Int\n  } deriving (Show, Generic, FromJSON, ToJSON)\n\ndata Content2 = Content2\n  { parts :: [Part]\n  , role :: Maybe String\n  } deriving (Show, Generic, FromJSON, ToJSON)\n\ndata Part = Part\n  { text :: String\n  } deriving (Show, Generic, FromJSON, ToJSON)\n\ndata PromptFeedback = PromptFeedback\n  { blockReason :: Maybe String\n  , safetyRatings :: Maybe [SafetyRating]\n  } deriving (Show, Generic, FromJSON, ToJSON)\n\ndata SafetyRating = SafetyRating\n  { category :: String\n  , probability :: String\n  } deriving (Show, Generic, FromJSON, ToJSON)\n

\n\n

There are two parts to function main. The first part defined the template for showing a HTML input form with embedded JavaScript:

main :: IO ()\nmain = do\n  apiKey <- getEnv \"GOOGLE_API_KEY\"\n  scotty 3000 $ do\n    get \"/\" $ do\n      html $ TL.pack\n        \"<!DOCTYPE html>\\\n        \\<html>\\\n        \\<head>\\\n        \\<title>Gemini Chat</title>\\\n        \\</head>\\\n        \\<body>\\\n        \\  <h1>Gemini Chat</h1>\\\n        \\  <form id='chat-form'>\\\n        \\    <input type='text' id='prompt' name='prompt' placeholder='Enter your pr\\\nompt'\\\n        \\           style='width: 70%; padding: 10px; font-size: 16px;'>\\\n        \\    <button type='submit'>Send</button>\\\n        \\  </form><br/><br/><h4>Response:</h4>\\\n        \\  <div id='response'></div>\\\n        \\  <script>\\\n        \\   const form = document.getElementById('chat-form');\\\n        \\   const responseDiv = document.getElementById('response');\\\n        \\   form.addEventListener('submit', async (event) => {\\\n        \\     event.preventDefault();\\\n        \\     const prompt = document.getElementById('prompt').value;\\\n        \\     try {\\\n        \\       const response = await fetch('/chat', {\\\n        \\         method: 'POST',\\\n        \\         headers: { 'Content-Type': 'application/json' },\\\n        \\         body: JSON.stringify({ prompt: prompt })\\\n        \\       });\\\n        \\       const data = await response.json();\\\n        \\       responseDiv.innerText = data.text;\\\n        \\     } catch (error) {\\\n        \\       console.error('Error:', error);\\\n        \\       responseDiv.innerText = 'Error occurred while fetching response';\\\n        \\     }\\\n        \\   });\\\n        \\  </script>\\\n        \\</body>\\\n        \\</html>\"\n

\n\n

The second part of function main defines the HTTP POST processing. When a user clicks the submit button on the web page. This handler is called by the JavaScript embedded on the web page:

    post \"/chat\" $ do\n      req <- jsonData :: ActionM GeminiRequest\n      liftIO $ putStrLn $ \"Received request: \" ++ show req\n      liftIO $ hFlush stdout\n\n      manager <- liftIO $ newManager tlsManagerSettings\n\n      initialRequest <- liftIO $ parseRequest \n        \"https://generativelanguage.googleapis.com/v1/models/gemini-pro:generateCont\\\nent\"\n\n      let geminiRequestBody = Aeson.object\n            [ (\"contents\", Aeson.Array $ V.singleton $ Aeson.object\n                [ (\"parts\", Aeson.Array $ V.singleton $ Aeson.object\n                    [ (\"text\", Aeson.String $ T.pack $ prompt req)\n                    ]\n                  )\n                ]\n              )\n            , (\"generationConfig\", Aeson.object\n                [ (\"temperature\", Aeson.Number 0.1)\n                , (\"maxOutputTokens\", Aeson.Number 800)\n\n                ]\n              )\n            ]\n\n      let request2 = initialRequest\n            { requestHeaders =\n                [ (\"Content-Type\", \"application/json\")\n                , (\"x-goog-api-key\", BS.pack apiKey)\n                ]\n            , method = \"POST\"\n            , requestBody = RequestBodyLBS $ Aeson.encode geminiRequestBody\n            }\n\n      liftIO $ putStrLn $ \"Request body: \" ++ show (Aeson.encode geminiRequestBody)\n      liftIO $ hFlush stdout\n\n      response2 <- liftIO $ httpLbs request2 manager\n      liftIO $ do\n        putStrLn $ \"Response status: \" ++ show (responseStatus response2)\n        putStrLn $ \"Response headers: \" ++ show (responseHeaders response2)\n        putStrLn $ \"Raw response: \" ++ show (responseBody response2)\n        hFlush stdout\n\n      let maybeGeminiResponse =\n          Aeson.decode (responseBody response2) :: Maybe GeminiResponse\n      \n      liftIO $ putStrLn $ \"Parsed response: \" ++ show maybeGeminiResponse\n      liftIO $ hFlush stdout\n\n      case maybeGeminiResponse of\n        Just geminiResponse -> do\n          case candidates geminiResponse of\n            (candidate:_) -> do\n              case parts (content candidate) of\n                (part:_) -> do\n                  liftIO $ putStrLn $ \"Sending response: \" ++ show part\n                  liftIO $ hFlush stdout\n                  json part\n                [] -> do\n                  liftIO $ putStrLn \"No parts in response\"\n                  status status500 >> Web.Scotty.text \"No content in response\"\n            [] -> do\n              liftIO $ putStrLn \"No candidates in response\"\n              status status500 >> Web.Scotty.text \"No candidates in response\"\n        Nothing -> do\n          liftIO $ putStrLn \"Failed to parse response\"\n          status status500 >> Web.Scotty.text \"Failed to parse Gemini response\"\n

\n\n

Web App Wrap Up and a Description of Scotty Web Framework

Scotty is a lightweight and expressive Haskell web framework that draws inspiration from Ruby’s Sinatra. It emphasizes simplicity and ease of use, making it a great choice for beginners and small projects. With Scotty, you can quickly define routes, handle requests, and generate responses with minimal boilerplate code. Its concise syntax and focus on core web development concepts allow developers to focus on building their application logic without getting bogged down in framework complexities.

Furthermore, Scotty leverages Haskell’s powerful type system to provide compile-time guarantees and enhanced code safety. This helps catch errors early in the development process and promotes maintainable code.

Scotty is built on top of the efficient WAI (Web Application Interface) standard, ensuring compatibility with a wide range of middleware and extensions. This allows developers to easily integrate Scotty with existing Haskell libraries and tools, extending its functionality to meet the needs of more complex applications.

This is a simple web app but I hope, dear reader, that it is a useful example and a good jumping off point for your own web app projects. This example also serves as an additional example in this book for using LLM APIs in Haskell applications.

Command Line Utility To Use the Google Gemini APIs

This example is similar to the example in the last chapter but here we build not a web application but a command line application to use the Google Gemini LLM APIs.

The directory haskell_tutorial_cookbook_examples/gemini_commandline contains the code for this example.

Before we look at the code let’s run the example:

$ gemini \"what is the square of pi?\"\nResponse:\n\nThe square of pi (π) is π multiplied by itself: π².  Since π is approximately 3.1415\\\n9, π² is approximately 9.8696.\n

\n\n

The executable file gemini is on my path because I copied the executable file to my personal bin directory:

$ cabal build\n$ find . -name gemini\n  ... output not shown\n$ cp ./dist-newstyle/build/aarch64-osx/ghc-9.4.8/gemini-0.1.0.0/x/gemini/build/gemin\\\ni/gemini ~/bin\n

\n\n

If you don’t want to permanently install this example on your laptop, then just run it with cabal:

$ cabal run gemini \"what is 11 + 23?\"\nResponse:\n\n11 + 23 = 34\n

\n\n

Here is a listing of the source file Main.hs (explanation after the code). This code is a Haskell program that interacts with Google’s Gemini AI model through its API. The program is structured to send prompts to Gemini and receive generated responses, implementing a command line interface for this interaction.

import Control.Monad.IO.Class (liftIO)\nimport System.Environment (getArgs, getEnv)\nimport qualified Data.Aeson as Aeson\nimport Data.Aeson (FromJSON, ToJSON)\nimport GHC.Generics\nimport Network.HTTP.Client.TLS (tlsManagerSettings)\nimport Network.HTTP.Client (newManager, httpLbs, parseRequest, Request(..), RequestB\\\nody(..), responseBody, responseStatus)\nimport Network.HTTP.Types.Status (statusCode)\nimport qualified Data.Text as T\nimport Data.Text.Encoding (encodeUtf8)\nimport qualified Data.Vector as V\n\ndata GeminiRequest = GeminiRequest\n  { prompt :: String\n  } deriving (Show, Generic, ToJSON)\n\ndata GeminiResponse = GeminiResponse\n  { candidates :: [Candidate]  -- Changed from choices to candidates\n  } deriving (Show, Generic, FromJSON)\n\ndata Candidate = Candidate\n  { content :: Content\n  } deriving (Show, Generic, FromJSON)\n\ndata Content = Content\n  { parts :: [Part]\n  } deriving (Show, Generic, FromJSON)\n\ndata Part = Part\n  { text :: String\n  } deriving (Show, Generic, FromJSON, ToJSON)\n\ndata PromptFeedback = PromptFeedback\n  { blockReason :: Maybe String\n  , safetyRatings :: Maybe [SafetyRating]\n  } deriving (Show, Generic, FromJSON, ToJSON)\n\ndata SafetyRating = SafetyRating\n  { category :: String\n  , probability :: String\n  } deriving (Show, Generic, FromJSON, ToJSON)\n\nmain :: IO ()\nmain = do\n  args <- getArgs\n  case args of\n    [] -> putStrLn \"Error: Please provide a prompt as a command line argument.\"\n    (arg:_) -> do  --  Extract the argument directly\n      apiKey <- getEnv \"GOOGLE_API_KEY\"\n\n      manager <- newManager tlsManagerSettings\n\n      initialRequest <- parseRequest \"https://generativelanguage.googleapis.com/v1/m\\\nodels/gemini-1.5-pro:generateContent\"\n\n      let geminiRequestBody = Aeson.object [\n            (\"contents\", Aeson.Array $ V.singleton $ Aeson.object [\n                (\"parts\", Aeson.Array $ V.singleton $ Aeson.object [\n                    (\"text\", Aeson.String $ T.pack  arg)\n                    ])\n            ]),\n            (\"generationConfig\", Aeson.object [\n                (\"temperature\", Aeson.Number 0.1),\n                (\"maxOutputTokens\", Aeson.Number 800)\n                ])\n            ]\n            \n      let request = initialRequest\n            { requestHeaders =\n                [ (\"Content-Type\", \"application/json\")\n                , (\"x-goog-api-key\", encodeUtf8 $ T.pack apiKey)\n                ]\n            , method = \"POST\"\n            , requestBody = RequestBodyLBS $ Aeson.encode geminiRequestBody\n            }\n\n      response <- httpLbs request manager\n      \n      let responseStatus' = responseStatus response\n\n      if statusCode responseStatus' == 200\n        then do\n        let maybeGeminiResponse =\n           Aeson.decode (responseBody response) :: Maybe GeminiResponse\n        case maybeGeminiResponse of\n          Just geminiResponse -> do\n            case candidates geminiResponse of\n              (candidate:_) -> do\n                case parts (content candidate) of\n                  (part:_) -> do  -- Changed text_ to _ since it's unused\n                    liftIO $ putStrLn $ \"Response:\\n\\n\" ++ text part\n                  [] -> do\n                    liftIO $ putStrLn \"Error: No parts in content\"\n              [] -> do\n                liftIO $ putStrLn \"Error: No candidates in response\"\n          Nothing -> do\n             liftIO $ putStrLn \"Error: Failed to parse response\"\n        else do\n        putStrLn $ \"Error: \" ++ show responseStatus'\n

\n\n

The first section of the code defines several data types using Haskell’s deriving mechanism for automatic JSON serialization/deserialization. These types (GeminiRequest, GeminiResponse, Candidate, Content, Part, PromptFeedback, and SafetyRating) mirror the JSON structure expected by the Gemini API. This demonstrates Haskell’s strong type system being used to ensure type-safe handling of API data structures.

The main function implements the program’s core logic: it retrieves a prompt from command-line arguments and an API key from environment variables, constructs an HTTP request to the Gemini API with proper headers and JSON body, and handles the response. The code uses monadic composition to handle the asynchronous nature of HTTP requests and includes error handling for various failure cases, such as missing arguments, API errors, or malformed responses. The response processing extracts the generated text from the nested data structure and prints it to the console. The code also includes configuration for the AI model’s parameters like temperature and maximum output tokens.

Command Line Tool to Use Local Ollama LLM Server

Ollama is a multi-platform tool for running Large Language Models (LLMs) on your laptop. Ollama is available for macOS, Linux, and Windows.

My personal preference is to run local models on my own computer, and just use commercial LLM APIs from OpenAI, Google, Anthropic, Mistral, etc. when local models are insufficient for specific tasks.

You can download Ollama from https://ollama.ai.

The directory haskell_tutorial_cookbook_examples/ollama_commandline contains the code for this example.

Before we look at the code, let’s look at how we use this command line tool:

cabal run ollama-client \"how much is 4 + 11 + 13?\"\ncabal run ollama-client \"write Python script to print 11th and 12th prime numbers\"\ncabal run ollama-client \"Write a Haskell hello world program\"\n

\n\n

I like to build the executable for this example using cabal build and then copy the executable file to my personal bin utility directory using these commands:

$ find . -name ollama-client\n$ cp ./dist-newstyle/build/aarch64-osx/ghc-9.4.8/ollama-client-0.1.0.0/x/ollama-clie\\\nnt/build/ollama-client/ollama-client ~/bin\n

\n\n

The find command is used to find the location of the executable generated by cabal build.

The example in this chapter is very similar to the example in the last chapter that uses Google’s Gemini model APIs. Currently Google makes using their APIs almost free but I still like running open weight models on my own laptop for privacy and security.

This example program is listed below. I will describe the code after the listing.

{-# LANGUAGE DuplicateRecordFields #-}\n{-# LANGUAGE OverloadedRecordDot #-}\n\nimport Control.Monad.IO.Class (liftIO)\nimport System.Environment (getArgs)\nimport qualified Data.Aeson as Aeson\nimport Data.Aeson (FromJSON, ToJSON)\nimport GHC.Generics\nimport Network.HTTP.Client (newManager, httpLbs, parseRequest, Request(..), RequestB\\\nody(..), responseBody, responseStatus, defaultManagerSettings)\nimport Network.HTTP.Types.Status (statusCode)\n\ndata OllamaRequest = OllamaRequest\n  { model :: String\n  , prompt :: String\n  , stream :: Bool\n  } deriving (Show, Generic, ToJSON)\n\ndata OllamaResponse = OllamaResponse\n  { model :: String\n  , created_at :: String\n  , response :: String  -- This matches the actual field name in the JSON\n  , done :: Bool\n  , done_reason :: String\n  } deriving (Show, Generic, FromJSON)\n\nmain :: IO ()\nmain = do\n  args <- getArgs\n  case args of\n    [] -> putStrLn \"Error: Please provide a prompt as a command-line argument.\"\n    (arg:_) -> do\n      manager <- newManager defaultManagerSettings\n\n      initialRequest <- parseRequest \"http://localhost:11434/api/generate\"\n\n      let ollamaRequestBody = OllamaRequest\n            { model = \"llama3.2:latest\"  -- You can change this to another  model\n            , prompt = arg\n            , stream = False\n            }\n\n      let request = initialRequest\n            { requestHeaders = [(\"Content-Type\", \"application/json\")]\n            , method = \"POST\"\n            , requestBody = RequestBodyLBS $ Aeson.encode ollamaRequestBody\n            }\n\n      httpResponse <- httpLbs request manager\n--    liftIO $ putStrLn $ \"httpResponse:\" ++ show httpResponse -- debug\n      \n      let responseStatus' = responseStatus httpResponse\n\n      if statusCode responseStatus' == 200\n        then do\n          let maybeOllamaResponse =\n                Aeson.decode (responseBody httpResponse) :: Maybe OllamaResponse\n          case maybeOllamaResponse of\n            Just ollamaResponse -> do\n              liftIO $ putStrLn $ \"Response:\\n\\n\" ++ ollamaResponse.response\n            Nothing -> do\n              liftIO $ putStrLn \"Error: Failed to parse response\"\n        else do\n          putStrLn $ \"Error: \" ++ show responseStatus'\n

\n\n

This Haskell code defines a program that interacts with a local HTTP API provided by Ollama to send a prompt to a language model and retrieve its response. The program uses several Haskell features and libraries, including OverloadedRecordDot for concise record field access and DuplicateRecordFields to allow multiple data types to share field names. The program is structured around two main data types, OllamaRequest and OllamaResponse, which represent the request and response payloads for the API. These data types are derived from the Generic type class, enabling automatic JSON serialization and deserialization using the Aeson library. The OllamaRequest type includes fields for specifying the model, the prompt, and whether the response should be streamed, while the OllamaResponse type captures the model’s response, creation time, and additional metadata.

The function main begins by reading command-line arguments using getArgs. If no arguments are provided, the program outputs an error message and terminates. Otherwise, it extracts the first argument as the prompt and proceeds to set up an HTTP client using newManager with default settings. The program constructs an HTTP POST request to the local API endpoint (http://localhost:11434/api/generate) using parseRequest. It then creates an instance of OllamaRequest with a hardcoded model name (llama3.2:latest), the user-provided prompt, and stream set to False. This request is serialized to JSON using Aeson.encode and included in the HTTP request body, with the appropriate Content-Type header.

The program sends the HTTP request using httpLbs, which performs the request and returns the response. It then checks the HTTP status code of the response using responseStatus and statusCode. If the status code is 200 (indicating success), the program attempts to decode the response body into an OllamaResponse object using Aeson.decode. If decoding is successful, it extracts and prints the response field from the OllamaResponse object using the OverloadedRecordDot extension for concise field access. If decoding fails, the program outputs an error message indicating that the response could not be parsed.

If the HTTP status code is not 200, the program outputs an error message with the status code. This error handling ensures that the user is informed of any issues with the API request or response. Overall, the program demonstrates how to use Haskell’s type system and libraries to build a robust and type-safe client for interacting with a JSON-based HTTP API. It also highlights the use of modern Haskell features, such as record dot syntax and generic programming, to simplify code and improve readability.

In general when you are writing code to access a LLM model using a REST interface, look for examples using the curl command, examine the returned JSON payload and write code to decode the JSON and extract the information you need. I started writing this example by finding the following curl example in the Ollama documentation and running it:

1 curl http://localhost:11434/api/generate -d '{\n2   \"model\": \"llama3.2\",\n3   \"prompt\": \"What is 3 + 5 + 11?\",\n4   \"stream\": false\n5 }'\n

\n\n

Using the Brave Search API

You need to sign up for a free or paid account on the Brave search page and set an environment variable to your assigned API key:

export BRAVE_SEARCH_API_KEY = BSAgQ-Nc5.....\n

\n\n

The Brave Search API allows you to access Brave Search results directly within your applications or services. It provides developers with the ability to harness the privacy-focused and independent search capabilities of Brave, returning results for web searches, news, videos, and more. To obtain an API key, simply create an account on the Brave Search API website and subscribe to either the free or one of the paid plans. The Brave Search API offers flexible pricing tiers, including a free option for testing and development, making it accessible to a wide range of users and projects. Currently you can call the API 2000 times a month on the free tier.

The library developed in this chapter is implemented in a single file BraveSearch.hs:

 1 {-# LANGUAGE OverloadedStrings #-}\n 2 {-# LANGUAGE RecordWildCards #-}\n 3 \n 4 module BraveSearch\n 5   ( getSearchSuggestions\n 6   ) where\n 7 \n 8 import Network.HTTP.Simple\n 9 import Data.Aeson\n10 import qualified Data.Text as T\n11 import Control.Exception (try)\n12 import Network.HTTP.Client (HttpException)\n13 import qualified Data.ByteString.Char8 as BS\n14 import qualified Data.ByteString.Lazy.Char8 as LBS\n15 \n16 data SearchResponse = SearchResponse\n17   { query :: QueryInfo\n18   , web :: WebResults\n19   } deriving (Show)\n20 \n21 data QueryInfo = QueryInfo\n22   { original :: T.Text\n23   } deriving (Show)\n24 \n25 data WebResults = WebResults\n26   { results :: [WebResult]\n27   } deriving (Show)\n28 \n29 data WebResult = WebResult\n30   { type_ :: T.Text\n31   , index :: Maybe Int\n32   , all :: Maybe Bool\n33   , title :: Maybe T.Text\n34   , url :: Maybe T.Text\n35   , description :: Maybe T.Text\n36   } deriving (Show)\n37 \n38 instance FromJSON SearchResponse where\n39   parseJSON = withObject \"SearchResponse\" $ \\v -> SearchResponse\n40     <$> v .: \"query\"\n41     <*> v .: \"web\"\n42 \n43 instance FromJSON QueryInfo where\n44   parseJSON = withObject \"QueryInfo\" $ \\v -> QueryInfo\n45     <$> v .: \"original\"\n46 \n47 instance FromJSON WebResults where\n48   parseJSON = withObject \"WebResults\" $ \\v -> WebResults\n49     <$> v .: \"results\"\n50 \n51 instance FromJSON WebResult where\n52   parseJSON = withObject \"WebResult\" $ \\v -> WebResult\n53     <$> v .: \"type\"\n54     <*> v .:? \"index\"\n55     <*> v .:? \"all\"\n56     <*> v .:? \"title\"\n57     <*> v .:? \"url\"\n58     <*> v .:? \"description\"\n59 \n60 getSearchSuggestions :: String -> String -> IO (Either String [T.Text])\n61 getSearchSuggestions apiKey query = do\n62   let url = \"https://api.search.brave.com/res/v1/web/search?q=\" ++\n63             query ++ \"&country=US&count=5\"\n64   \n65   request <- parseRequest url\n66   let requestWithHeaders = setRequestHeader \"Accept\" [\"application/json\"]\n67                          $ setRequestHeader \"X-Subscription-Token\" [BS.pack apiKey]\n68                          $ request\n69   \n70   result <- try $ httpLBS requestWithHeaders\n71   \n72   case result of\n73     Left e -> return $ Left $ \"Network error: \" ++ show (e :: HttpException)\n74     Right response -> do\n75       let statusCode = getResponseStatusCode response\n76       if statusCode /= 200\n77         then return $ Left $ \"HTTP error: \" ++ show statusCode\n78         else do\n79           let body = getResponseBody response\n80           case eitherDecode body of\n81             Left err -> return $ Left $ \"JSON parsing error: \" ++ err\n82             Right searchResponse@SearchResponse{..} -> do\n83               let originalQuery = original query\n84                   webResults = results web\n85               let suggestions = \"Original Query: \" <>\n86                   originalQuery : map formatResult webResults\n87               return $ Right suggestions\n88 \n89 formatResult :: WebResult -> T.Text\n90 formatResult WebResult{..} =\n91   let titleText = maybe \"N/A\" (\"Title: \" <>) title\n92       urlText = maybe \"N/A\" (\"URL: \" <>) url\n93       descText = maybe \"N/A\" (\"Description: \" <>) (fmap (T.take 100) description)\n94   in T.intercalate \" | \" [titleText, urlText, descText]\n

\n\n

Haskell Code Description for Brave Search Suggestions

This Haskell code implements a function, getSearchSuggestions, that fetches search suggestions from the Brave Search API.

Functionality:

\n
\ngetSearchSuggestions:\n
\n
\n
- \nTakes an API key and a search query as input.\n
- \nConstructs a URL to send a request to the Brave Search API, specifying the query, country, and result count.\n
- \nSets up the HTTP request with necessary headers, including the API key.\n
- \nMakes the request and handles potential network errors.\n
- \nChecks the response status code. If it’s 200 (OK), proceeds to parse the JSON response.\n
- \nExtracts search results and formats them, including the original query.\n
- \nReturns either an error message (if something went wrong) or a list of formatted search suggestions.\n
\n
\n

Key Features:

\n
\nData Types:\n
\n
\n
- \nDefines data types to model the JSON structure of the Brave Search API response, including SearchResponse, QueryInfo, WebResults, and WebResult.\n
\n
\n
\n
\nJSON Parsing:\n
\n
\n
- \nUses the aeson library to parse the JSON response into the defined data types.\n
\n
\n
\n
\nError Handling:\n
\n
\n
- \nEmploys try from the Control.Exception module to gracefully handle potential network errors during the HTTP request.\n
\n
\n
\n
\nHTTP Request:\n
\n
\n
- \nUtilizes the Network.HTTP.Simple library to make the HTTP request to the Brave Search API.\n
\n
\n
\n
\nFormatting:\n
\n
\n
- \nThe formatResult function formats each search result into a user-friendly string, including the title, URL, and a shortened description.\n
\n
\n

Libraries Used:

\nNetwork.HTTP.Simple - For making HTTP requests.\n
\nData.Aeson - For JSON parsing and encoding.\n
\nData.Text - For efficient text handling.\n
\nControl.Exception - For error handling.\n
\nNetwork.HTTP.Client - For additional HTTP functionalities.\n
\nData.ByteString.Char8 and Data.ByteString.Lazy.Char8 - For working with byte strings.\n

Language Extensions:

\nOverloadedStrings - Allows the use of string literals as Text values.\n
\nRecordWildCards - Enables convenient access to record fields using wildcards.\n

Overall:

This code provides a basic but functional way to interact with the Brave Search API to retrieve and format search suggestions. It demonstrates good practices in Haskell programming, including data modeling, error handling, and the use of relevant libraries.

Here is an example Main.hs file to use this library:

 1 {-# LANGUAGE OverloadedStrings #-}\n 2 \n 3 module Main where\n 4 \n 5 import BraveSearch (getSearchSuggestions)\n 6 import System.Environment (getEnv)\n 7 import qualified Data.Text as T\n 8 import qualified Data.Text.IO as TIO\n 9 \n10 main :: IO ()\n11 main = do\n12   -- Get the API key from the environment variable\n13   apiKey <- getEnv \"BRAVE_SEARCH_API_KEY\"\n14   \n15   -- Prompt the user for a search query\n16   TIO.putStrLn \"Enter a search query:\"\n17   query <- TIO.getLine\n18   \n19   -- Call the function to get search suggestions\n20   result <- getSearchSuggestions apiKey (T.unpack query)\n21   \n22   case result of\n23     Left err -> TIO.putStrLn $ \"Error: \" <> T.pack err\n24     Right suggestions -> do\n25       TIO.putStrLn \"Search suggestions:\"\n26       mapM_ (TIO.putStrLn . (\"- \" <>)) suggestions\n

\n\n

Test Code Explanation

The code interacts with the BraveSearch module to demonstrate how to fetch and display search suggestions from the Brave Search API.

Code Breakdown

\n
\n
Imports
\n
\n
\n
- \n
  \nIt imports:\n
  \n
  \n
  - \nBraveSearch to use the getSearchSuggestions function.\n
  - \nSystem.Environment to get the API key from an environment variable.\n
  - \nData.Text and Data.Text.IO for working with text input and output.\n
  \n
  \n
\n
\n
\n
\n
main Function
\n
\n
\n
1. \n
  \n
  Get API Key
  \n
  \n
  \n
  - \nIt uses getEnv \"BRAVE_SEARCH_API_KEY\" to retrieve the Brave Search API key from an environment variable named BRAVE_SEARCH_API_KEY. This assumes you have set this environment variable in your system before running the code.\n
  \n
  \n
2. \n
  \n
  Prompt for Query
  \n
  \n
  \n
  - \nIt prints the message “Enter a search query:” to the console, prompting the user to input a search term.\n
  - \nIt reads the user’s input using TIO.getLine and stores it in the query variable.\n
  \n
  \n
3. \n
  \n
  Fetch Search Suggestions
  \n
  \n
  \n
  - \nIt calls the getSearchSuggestions function from the BraveSearch module, passing the API key and the user’s query (converted from Text to String using T.unpack).\n
  - \nIt stores the result of this call in the result variable.\n
  \n
  \n
4. \n
  \n
  Handle Result
  \n
  \n
  \n
  - \n
    \n
    It uses a case expression to handle the two possible outcomes of the getSearchSuggestions call:
    \n
    \n
    \n
    \n
    \n
    \n
    Left err:
    \n
    \n
    \n
    \n
    \nIf there was an error (e.g., network issue, HTTP error, JSON parsing error), it prints the error message prefixed with “Error:”.\n
    \n
    \n
    \n
    \n
    \n
    \n
    Right suggestions:
    \n
    \n
    \n
    \n
    \n
    \nIf the call was successful and returned a list of search suggestions:\n
    \n
    \n
    \n
    \nPrints “Search suggestions:” to the console.\n
    \n
    \nUses mapM_ to iterate over the suggestions list and print each suggestion in the following format: - suggestion text\n
    \n
    \n
    \n
    \n
    \n
    \n
    \n
    \n
    \n
  \n
  \n
\n
\n

This test code provides a basic example of how to use the getSearchSuggestions function from the BraveSearch module.

Here is the output:

$ cabal run     \nEnter a search query:\nfind a consultant for AI and common lisp, and the semantic web\nSearch suggestions:\n- Original Query: find a consultant for AI and common lisp, and the semantic web\n- Title: Mark Watson: AI Practitioner and Lisp Hacker | URL: https://markwatson.com/\\\n | Description: I am the author of 20+ books on Artificial Intelligence, <strong>Com\nmon</strong> <strong>Lisp</stron\n- Title: Lisp (programming language) - Wikipedia | URL: https://en.wikipedia.org/wik\\\ni/Lisp_(programming_language) | Description: Scheme is a statically scoped and prope\nrly tail-recursive dialect of <strong>the</strong> <strong>Li\n- Title: The Lisp approach to AI (Part 1). If you are a programmer that reads… | by \\\nSebastian Valencia | AI Society | Medium | URL: https://medium.com/ai-society/the-li\nsp-approach-to-ai-part-1-a48c7385a913 | Description: If you are a programmer that re\nads about the history and random facts of this lovely craft\n

\n\n

Text Processing

In my work in data science and machine learning, processing text is a core activity. I am a practitioner, not a research scientist, and in a practical sense, I spend a fair amount of time collecting data (e.g., web scraping and using semantic web/linked data sources), cleaning it, and converting it to different formats.

We will cover three useful techniques: parsing and using CSV (comma separated values) spreadsheet files, parsing and using JSON data, and cleaning up natural language text that contains noise characters.

CSV Spreadsheet Files

The comma separated values (CSV) format is a plain text format that all spreadsheet applications support. The following example illustrates two techniques that we haven’t covered yet:

\nExtracting values from the Either type.\n
\nUsing destructuring to concisely extract parts of a list.\n

The Either type Either a b contains either a Left a or a Right b value and is usually used to return an error in Left or a value in Right. We will using the Data.Either.Unwrap module to unwrap the Right part of a call to the Text.CSV.parseCSVFromFile function that reads a CSV file and returns a Left error or the data in the spreadsheet in a list as the Right value.

The destructuring trick in line 15 in the following listing lets us separate the head and rest of a list in one operation; for example:

*TestCSV> let z = [1,2,3,4,5]\n*TestCSV> z\n[1,2,3,4,5]\n*TestCSV> let x:xs = z\n*TestCSV> x\n1\n*TestCSV> xs\n[2,3,4,5]\n

\n\n

Here is how to read a CSV file:

module TestCSV where\n\nimport Text.CSV (parseCSVFromFile, CSV)\nimport Data.Either.Unwrap (fromRight)\n\nreadCsvFile :: FilePath -> CSV\nreadCsvFile fname = do\n  c <- parseCSVFromFile fname\n  return $ fromRight c\n\nmain = do\n  c <- readCsvFile \"test.csv\"\n  print  c  -- includes header and data rows\n  print $ map head c  -- print header\n  let header:rows = c -- destructure\n  print header\n  print rows\n

\n\n

Function readCsvFile reads from a file and returns a CSV. What is a CSV type? You could search the web for documentation, but dear reader, if you have worked this far learning Haskell, by now you know to rely on the GHCi repl:

*TestCSV> :i CSV\ntype CSV = [Text.CSV.Record]    -- Defined in ‘Text.CSV’\n*TestCSV> :i Text.CSV.Record\ntype Text.CSV.Record = [Text.CSV.Field]     -- Defined in ‘Text.CSV’\n*TestCSV> :i Text.CSV.Field\ntype Text.CSV.Field = String    -- Defined in ‘Text.CSV’\n

\n\n

So, a CSV is a list of records (rows in the spreadsheet file), each record is a list of fields (i.e., a string value).

The output when reading the CVS file test.csv is:

Prelude> :l TestCSV\n[1 of 1] Compiling TestCSV          ( TestCSV.hs, interpreted )\nOk, modules loaded: TestCSV.\n*TestCSV> main\n[[\"name\",\" email\",\" age\"],[\"John Smith\",\" jsmith@acmetools.com\",\" 41\"],[\"June Jones\"\\\n,\" jj@acmetools.com\",\" 38\"]]\n[\"name\",\"John Smith\",\"June Jones\"]\n[\"name\",\" email\",\" age\"]\n[[\"John Smith\",\" jsmith@acmetools.com\",\" 41\"],[\"June Jones\",\" jj@acmetools.com\",\" 38\\\n\"]]\n

\n\n

JSON Data

JSON is the native data format for the Javascript language and JSON has become a popular serialization format for exchanging data between programs on a network. In this section I will demonstrate serializing a Haskell type to a string with JSON encoding and then perform the opposite operation of deserializing a string containing JSON encoded data back to an object.

The first example uses the module Text.JSON.Generic (from the json library) and the second example uses module Data.Aeson (from the aeson library).

In the first example, we set the language type to include DeriveDataTypeable so a new type definition can simply derive Typeable which allows the compiler to generate appropriate encodeJSON and decodeJSON functions for the type Person we define in the example:

 1 {-# LANGUAGE DeriveDataTypeable #-}\n 2 \n 3 module TestTextJSON where\n 4 \n 5 import Text.JSON.Generic\n 6           \n 7 data Person = Person {name::String, email::String}\n 8                      deriving (Show, Data, Typeable)\n 9 \n10 main = do\n11   let a = encodeJSON $ Person \"Sam\" \"sam@a.com\"\n12   print a\n13   let d = (decodeJSON a :: Person)\n14   print d\n15   print $ name d\n16   print $ email d\n

\n\n

Notice that in line 13 that I specified the expected type in the decodeJSON call. This is not strictly required, the Haskell GHC compiler knows what to do in this case. I specified the type for code readability. The Haskell compiler wrote the name and email functions for me and I use these functions in lines 15 and 16 to extract these fields. Here is the output from running this example:

Prelude> :l TestTextJSON.hs \n[1 of 1] Compiling TestTextJSON     ( TestTextJSON.hs, interpreted )\nOk, modules loaded: TestTextJSON.\n*TestTextJSON> main\n\"{\\\"name\\\":\\\"Sam\\\",\\\"email\\\":\\\"sam@a.com\\\"}\"\nPerson {name = \"Sam\", email = \"sam@a.com\"}\n\"Sam\"\n\"sam@a.com\"\n

\n\n

The next example uses the Aeson library and is similar to this example.

Using Aeson, we set a language type DeriveGeneric and in this case have the Person class derive Generic. The School of Haskell has an excellent Aeson tutorial that shows a trick I use in this example: letting the compiler generate required functions for types FromJSON and ToJSON as seen in lines 12-13.

 1 {-# LANGUAGE DeriveGeneric #-}\n 2 \n 3 module TestJSON where\n 4 \n 5 import Data.Aeson\n 6 import GHC.Generics\n 7 import Data.Maybe\n 8 \n 9 data Person = Person {name::String, email::String } deriving (Show, Generic)\n10 \n11 -- nice trick from School Of Haskell tutorial on Aeson:\n12 instance FromJSON Person  -- DeriveGeneric language setting allows\n13 instance ToJSON Person    -- automatic generation of instance of\n14                           -- types deriving Generic.\n15 \n16 main = do\n17   let a = encode $ Person \"Sam\" \"sam@a.com\"\n18   print a\n19   let (Just d) = (decode a :: Maybe Person)\n20   print d\n21   print $ name d\n22   print $ email d\n

\n\n

I use a short cut in line 19, assuming that the Maybe object returned from decode (which the compiler wrote automatically for the type FromJSON) contains a Just value instead of an empty Nothing value. So in line 19 I directly unwrap the Just value.

Here is the output from running this example:

1 Prelude> :l TestAESON.hs \n2 [1 of 1] Compiling TestJSON         ( TestAESON.hs, interpreted )\n3 Ok, modules loaded: TestJSON.\n4 *TestJSON> main\n5 \"{\\\"email\\\":\\\"sam@a.com\\\",\\\"name\\\":\\\"Sam\\\"}\"\n6 Person {name = \"Sam\", email = \"sam@a.com\"}\n7 \"Sam\"\n8 \"sam@a.com\"\n

\n\n

Line 5 shows the result of printing the JSON encoded string value created by the call to encode in line 17 of the last code example. Line 6 shows the decoded value of type Person, and lines 7 and 8 show the inner wrapped values in the Person data.

Cleaning Natural Language Text

I spend a lot of time working with text data because I have worked on NLP (natural language processing) projects for over 25 years. We will jump into some interesting NLP applications in the next chapter. I will finish this chapter with strategies for cleaning up text which is often a precursor to performing NLP.

You might be asking why we would need to clean up text. Here are a few common use cases:

\nText fetched from the web frequently contains garbage characters.\n
\nSome types of punctuation need to be removed.\n
\nStop words (e.g., the, a, but, etc.) need to be removed.\n
\nSpecial unicode characters are not desired.\n
\nSometimes we want white space around punctuation to make tokenizing text easier.\n

Notice the module statement on line 1 of the following listing: I am exporting functions cleanText and removeStopWords so they will be visible and available for use by any other modules that import this module. In line 6 we import intercalate which constructs a string from a space character and an [String] (i.e., a list of strings); here is an example where instead of adding a space character between the strings joined together, I add “*” characters:

*CleanText> intercalate \"*\" [\"the\", \"black\", \"cat\"]\n\"the*black*cat\"\n

\n\n

The function cleanText removes garbage characters and makes sure that any punctuation characters are surrounded by white space (this makes it easier, for example, to determine sentence boundaries). Function removeStopWords removes common words like “a”, “the”, etc. from text.

 1 module CleanText (cleanText, removeStopWords)  where\n 2 \n 3 import Data.List.Split (splitOn)\n 4 import Data.List (intercalate)\n 5 import Data.Char as C\n 6 import Data.List.Utils (replace)\n 7 \n 8 noiseCharacters = ['[', ']', '{', '}', '\\n', '\\t', '&', '^', \n 9                    '@', '%', '$', '#', ',']\n10 \n11 substituteNoiseCharacters :: [Char] -> [Char]\n12 substituteNoiseCharacters =\n13   map (\\x -> if elem x noiseCharacters then ' ' else x)\n14 \n15 cleanText s =\n16   intercalate\n17    \" \" $\n18    filter\n19      (\\x -> length x > 0) $\n20      splitOn \" \" $ substituteNoiseCharacters $\n21        (replace \".\" \" . \"\n22         (replace \",\" \" , \" \n23          (replace \";\" \" ; \" s)))\n24 \n25 stopWords = [\"a\", \"the\", \"that\", \"of\", \"an\"]\n26 \n27 toLower' :: [Char] -> [Char]\n28 toLower' s = map (\\x -> if isLower x then x else (C.toLower x)) s\n29 \n30 removeStopWords :: String -> [Char]\n31 removeStopWords s =\n32   intercalate\n33      \" \" $\n34     filter\n35       (\\x -> notElem (toLower' x) stopWords) $\n36       words s\n37 \n38 main = do\n39   let ct = cleanText \"The[]@] cat, and all dog, escaped&^. They were caught.\"\n40   print ct\n41   let nn = removeStopWords ct\n42   print nn\n

\n\n

This example should be extended with additional noise characters and stop words, depending on your application. The function cleanText simply uses substring replacements.

Let’s look more closely at removeStopWords that takes a single argument s, which is expected to be a string. removeStopWords uses a combination of several functions to remove stop words from the input string. The function words is used to split the input string s into a list of words.\nThen, the function filter is used to remove any words that match a specific condition. Here the condition is defined as a lambda function, which is passed as the first argument to the filter function. The lambda function takes a single argument x and returns a Boolean value indicating whether the word should be included in the output or not. The lambda function uses function notElem to check whether the lowercased version of the word x is present in a predefined list of stop words. Finally, we use the function intercalate to join the remaining words back into a single string. The first argument to function ** intercalate** is the separator that should be used to join the words, in this case, it’s a single space.

Here is the output from this example:

*TestCleanText> :l CleanText.hs \n[1 of 1] Compiling TestCleanText    ( CleanText.hs, interpreted )\nOk, modules loaded: TestCleanText.\n*TestCleanText> main\n\"The cat and all dog escaped . They were caught .\"\n\"cat dog escaped . They were caught .\"\n

\n\n

We will continue working with text in the next chapter.

Natural Language Processing Tools

The tools developed in this chapter are modules you can reuse in your programs. We will develop a command line program that reads a line of text from STDIN and writes semantic information as output to STDOUT. I have used this in a Ruby program by piping input text data to a forked process and reading the output which is a semantic representation of the input text.

Note: we previously saw a small application of the OpenAI completion LLMs to find place names in input text. We could replace most of the examples in this chapter with calls to a LLM completion API with NLP specific prompts.

We will be using this example as an external dependency to a later example in the chapter Knowledge Graph Creator.

A few of the data files I provide in this example are fairly large. As an example the file PeopleDbPedia.hs which builds a map from people’s names to the Wikipedia/DBPedia URI for information about them, is 2.5 megabytes in size. The first time you run stack build in the project directory it will take a while, so you might want to start building the project in the directory NlpTool and let it run while you read this chapter.

Here are three examples using the NlpTool command line application developed in this chapter:

Enter text (all on one line)\nCanada and England signed a trade deal.\ncategory:   economics\nsummary:    Canada and England signed a trade deal. \ncountries:  [[\"Canada\",\"<http://dbpedia.org/resource/Canada>\"],\n             [\"England\",\"<http://dbpedia.org/resource/England>\"]]\nEnter text (all on one line)\nPresident George W Bush asked Congress for permission to invade Iraq.\ncategory:   news_war\nsummary:    President George W Bush asked Congress for permission to invade Iraq. \npeople: [[\"George W Bush\",\"<http://dbpedia.org/resource/George_W._Bush>\"]]\ncountries:  [[\"Iraq\",\"\"]]\nEnter text (all on one line)\nThe British government is facing criticism from business groups over statements sugg\\\nesting the U.K. is heading for a hard divorce from the European Union — and pressure\n from lawmakers who want Parliament to have a vote on the proposed exit terms. The g\novernment's repeated emphasis on controlling immigration sent out \"signs that the do\nor is being closed, to an extent, on the open economy, that has helped fuel investme\nnt,\" the head of employers' group the Confederation of British Industry, Carolyn Fai\nrbairn, said in comments published Monday. Prime Minister Theresa May said last week\n that Britain would seek to retain a close relationship with the 28-nation bloc, wit\nh continued free trade in goods and services. But she said the U.K. wouldn't cede co\nntrol over immigration, a conflict with the EU's principle of free movement among me\nmber states.\ncategory:   economics\nsummary:    Prime Minister Theresa May said last week that Britain would seek to ret\\\nain a close relationship with the 28-nation bloc, with continued free trade in goods\n and services. \n

\n\n

credit: news text from abcnews.com

Resolve Entities in Text to DBPedia URIs

The code for this application is in the directory NlpTool.

The software and data in this chapter can be used under the terms of either the GPL version 3 license or the Apache 2 license.

There are several automatically generated Haskell formatted data files that I created using Ruby scripts operating the Wikipedia data. For the purposes of this book I include these data-specific files for your use and enjoyment but we won’t spend much time discussing them. These files are:

\nBroadcastNetworkNamesDbPedia.hs\n
\nCityNamesDbpedia.hs\n
\nCompanyNamesDbpedia.hs\n
\nCountryNamesDbpedia.hs\n
\nPeopleDbPedia.hs\n
\nPoliticalPartyNamesDbPedia.hs\n
\nTradeUnionNamesDbPedia.hs\n
\nUniversityNamesDbPedia.hs\n

As an example, let’s look at a small sample of data in PeopleDbPedia.hs:

1 module PeopleDbPedia (peopleMap) where\n2 \n3 import qualified Data.Map as M\n4 \n5 peopleMap = M.fromList [\n6   (\"Aaron Sorkin\", \"<http://dbpedia.org/resource/Aaron_Sorkin>\"),\n7   (\"Bill Clinton\", \"<http://dbpedia.org/resource/Bill_Clinton>\"),\n8   (\"George W Bush\", \"<http://dbpedia.org/resource/George_W_Bush>\"),\n

\n\n

There are 35,146 names in the file PeopleDbPedia.hs. I have built for eight different types of entity names: Haskell maps that take entity names (String) and maps the entity names into relevant DBPedia URIs. Simple in principle, but a lot of work preparing the data. As I mentioned, we will use these data-specific files to resolve entity references in text.

The next listing shows the file Entities.hs. In lines 5-7 I import the entity mapping files I just described. In this example and later code I make heavy use of the Data.Map and Data.Set modules in the collections library (see the NlpTools.cabal file).

The operator isSubsetOf defined in line 39 tests to see if a value is contained in a collection. The built-in function all applies a function or operator to all elements in a collection and returns a true value if the function or operator returns true applied to each element in the collection.

The local utility function namesHelper defined in lines 41-53 is simpler than it looks. The function filter in line 42 applies the inline function in lines 43-45 (this function returns true for Maybe values that contain data) to a second list defined in lines 48-55. This second list is calculated by mapping an inline function over the input argument ngrams. The inline function looks up an ngram in a DBPedia map (passed as the second function argument) and returns the lookup value if it is not empty and if it is empty looks up the same ngram in a word map (last argument to this function).

The utility function namesHelper is then used to define functions to recognize company names, country names, people names, city names, broadcast network names, political party names, trade union names, and university names:

  1 -- Copyright 2014 by Mark Watson. All rights reserved. The software and data in this\\\n  2  project can be used under the terms of either the GPL version 3 license or the Apac\n  3 he 2 license.\n  4 \n  5 module Entities (companyNames, peopleNames,\n  6                  countryNames, cityNames, broadcastNetworkNames,\n  7                  politicalPartyNames, tradeUnionNames, universityNames) where\n  8 \n  9 import qualified Data.Map as M\n 10 import qualified Data.Set as S\n 11 import Data.Char (toLower)\n 12 import Data.List (sort, intersect, intersperse)\n 13 import Data.Set (empty)\n 14 import Data.Maybe (isJust)\n 15 \n 16 import Utils (splitWords, bigram, bigram_s, splitWordsKeepCase,\n 17               trigram, trigram_s, removeDuplicates)\n 18 \n 19 import FirstNames (firstNames)\n 20 import LastNames (lastNames)\n 21 import NamePrefixes (namePrefixes)\n 22 \n 23 import PeopleDbPedia (peopleMap)\n 24 \n 25 import CountryNamesDbpedia (countryMap)\n 26 import CountryNames (countryNamesOneWord, countryNamesTwoWords, countryNamesThreeWor\\\n 27 ds)\n 28 \n 29 import CompanyNamesDbpedia (companyMap)\n 30 import CompanyNames (companyNamesOneWord, companyNamesTwoWords, companyNamesThreeWor\\\n 31 ds)\n 32 import CityNamesDbpedia (cityMap)\n 33  \n 34 import BroadcastNetworkNamesDbPedia (broadcastNetworkMap)\n 35 import PoliticalPartyNamesDbPedia (politicalPartyMap)\n 36 import TradeUnionNamesDbPedia (tradeUnionMap)\n 37 import UniversityNamesDbPedia (universityMap)\n 38 \n 39 xs `isSubsetOf` ys = all (`elem` ys) xs\n 40     \n 41 namesHelper ngrams dbPediaMap wordMap =\n 42   filter \n 43     (\\x -> case x of\n 44          (_, Just x) -> True\n 45          _ -> False) $\n 46     map (\\ngram -> (ngram,\n 47                 let v = M.lookup ngram dbPediaMap in\n 48                 if isJust v\n 49                    then return (ngram, v)\n 50                    else if S.member ngram wordMap\n 51                            then Just (ngram, Just \"\")\n 52                            else Nothing))\n 53         ngrams   \n 54 \n 55 helperNames1W = namesHelper\n 56 \n 57 helperNames2W wrds = namesHelper (bigram_s wrds)\n 58     \n 59 helperNames3W wrds =  namesHelper (trigram_s wrds)\n 60 \n 61 companyNames wrds =\n 62   let cns = removeDuplicates $ sort $\n 63               helperNames1W wrds companyMap companyNamesOneWord ++\n 64               helperNames2W wrds companyMap companyNamesTwoWords ++\n 65               helperNames3W wrds companyMap companyNamesThreeWords in\n 66   map (\\(s, Just (a,Just b)) -> (a,b)) cns\n 67   \n 68 countryNames wrds =\n 69   let cns = removeDuplicates $ sort $\n 70               helperNames1W wrds countryMap countryNamesOneWord ++\n 71               helperNames2W wrds countryMap countryNamesTwoWords ++\n 72               helperNames3W wrds countryMap countryNamesThreeWords in\n 73   map (\\(s, Just (a,Just b)) -> (a,b)) cns\n 74 \n 75 peopleNames wrds =\n 76   let cns = removeDuplicates $ sort $\n 77               helperNames1W wrds peopleMap Data.Set.empty ++\n 78               helperNames2W wrds peopleMap Data.Set.empty ++\n 79               helperNames3W wrds peopleMap Data.Set.empty in\n 80   map (\\(s, Just (a,Just b)) -> (a,b)) cns\n 81 \n 82 cityNames wrds =\n 83   let cns = removeDuplicates $ sort $\n 84               helperNames1W wrds cityMap Data.Set.empty ++\n 85               helperNames2W wrds cityMap Data.Set.empty ++\n 86               helperNames3W wrds cityMap Data.Set.empty in\n 87   map (\\(s, Just (a,Just b)) -> (a,b)) cns\n 88 \n 89 broadcastNetworkNames wrds =\n 90   let cns = removeDuplicates $ sort $\n 91               helperNames1W wrds broadcastNetworkMap Data.Set.empty ++\n 92               helperNames2W wrds broadcastNetworkMap Data.Set.empty ++\n 93               helperNames3W wrds broadcastNetworkMap Data.Set.empty in\n 94   map (\\(s, Just (a,Just b)) -> (a,b)) cns\n 95 \n 96 politicalPartyNames wrds =\n 97   let cns = removeDuplicates $ sort $\n 98               helperNames1W wrds politicalPartyMap Data.Set.empty ++\n 99               helperNames2W wrds politicalPartyMap Data.Set.empty ++\n100               helperNames3W wrds politicalPartyMap Data.Set.empty in\n101   map (\\(s, Just (a,Just b)) -> (a,b)) cns\n102 \n103 tradeUnionNames wrds =\n104   let cns = removeDuplicates $ sort $\n105               helperNames1W wrds tradeUnionMap Data.Set.empty ++\n106               helperNames2W wrds tradeUnionMap Data.Set.empty ++\n107               helperNames3W wrds tradeUnionMap Data.Set.empty in\n108   map (\\(s, Just (a,Just b)) -> (a,b)) cns\n109 \n110 universityNames wrds =\n111   let cns = removeDuplicates $ sort $\n112              helperNames1W wrds universityMap Data.Set.empty ++\n113              helperNames2W wrds universityMap Data.Set.empty ++\n114              helperNames3W wrds universityMap Data.Set.empty in\n115   map (\\(s, Just (a,Just b)) -> (a,b)) cns\n116 \n117 \n118 main = do\n119     let s = \"As read in the San Francisco Chronicle, the company is owned by John Sm\\\n120 ith, Bill Clinton, Betty Sanders, and Dr. Ben Jones. Ben Jones and Mr. John Smith ar\n121 e childhood friends who grew up in Brazil, Canada, Buenos Aires, and the British Vir\n122 gin Islands. Apple Computer relased a new version of OS X yesterday. Brazil Brazil B\n123 razil. John Smith bought stock in ConocoPhillips, Heinz, Hasbro, and General Motors,\n124  Fox Sports Radio. I listen to B J Cole. Awami National Party is a political party. \n125 ALAEA is a trade union. She went to Brandeis University.\"\n126     --print $ humanNames s\n127     print $ peopleNames $ splitWordsKeepCase s\n128     print $ countryNames $ splitWordsKeepCase s\n129     print $ companyNames $ splitWordsKeepCase s\n130     print $ cityNames $ splitWordsKeepCase s\n131     print $ broadcastNetworkNames $ splitWordsKeepCase s\n132     print $ politicalPartyNames $ splitWordsKeepCase s\n133     print $ tradeUnionNames $ splitWordsKeepCase s\n134     print $ universityNames $ splitWordsKeepCase s\n

\n\n

The following output is generated by running the test main function defined at the bottom of the file app/NlpTool.hs:

$ stack build --fast --exec NlpTool-exe\nBuilding all executables for `NlpTool' once. After a successful build of all of them\\\n, only specified executables will be rebuilt.\nNlpTool> build (lib + exe)\nPreprocessing library for NlpTool-0.1.0.0..\nBuilding library for NlpTool-0.1.0.0..\nPreprocessing executable 'NlpTool-exe' for NlpTool-0.1.0.0..\nBuilding executable 'NlpTool-exe' for NlpTool-0.1.0.0..\n[1 of 2] Compiling Main\n[2 of 2] Compiling Paths_NlpTool\nLinking .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/NlpTool-exe/NlpTool-exe ...\nNlpTool> copy/register\nInstalling library in /Users/markw/GITHUB/haskell_tutorial_cookbook_examples_private\\\n_new_edition/NlpTool/.stack-work/install/x86_64-osx/7a2928fbf8188dcb20f165f77b37045a\n5c413cc7f63913951296700a6b7e292d/8.6.5/lib/x86_64-osx-ghc-8.6.5/NlpTool-0.1.0.0-DXKb\nucyA0S0AKOAcZGDl2H\nInstalling executable NlpTool-exe in /Users/markw/GITHUB/haskell_tutorial_cookbook_e\\\nxamples_private_new_edition/NlpTool/.stack-work/install/x86_64-osx/7a2928fbf8188dcb2\n0f165f77b37045a5c413cc7f63913951296700a6b7e292d/8.6.5/bin\nRegistering library for NlpTool-0.1.0.0..\nEnter text (all on one line)\nAs read in the San Francisco Chronicle, the company is owned by John Smith, Bill Cli\\\nnton, Betty Sanders, and Dr. Ben Jones. Ben Jones and Mr. John Smith are childhood f\nriends who grew up in Brazil, Canada, Buenos Aires, and the British Virgin Islands. \nApple Computer relased a new version of OS X yesterday. Brazil Brazil Brazil. John S\nmith bought stock in ConocoPhillips, Heinz, Hasbro, and General Motors, Fox Sports R\nadio. I listen to B J Cole. Awami National Party is a political party. ALAEA is a tr\nade union. She went to Brandeis University.\ncategory:   news_politics\nsummary:    ALAEA is a trade union. Apple Computer relased a new version of OS X yes\\\nterday.\npeople: [[\"B J Cole\",\"<http://dbpedia.org/resource/B._J._Cole>\"]]\ncompanies:  [[\"Apple\",\"<http://dbpedia.org/resource/Apple>\"],[\"ConocoPhillips\",\"<htt\\\np://dbpedia.org/resource/ConocoPhillips>\"],[\"Hasbro\",\"<http://dbpedia.org/resource/H\nasbro>\"],[\"Heinz\",\"<http://dbpedia.org/resource/Heinz>\"],[\"San Francisco Chronicle\",\n\"<http://dbpedia.org/resource/San_Francisco_Chronicle>\"]]\ncountries:  [[\"Brazil\",\"<http://dbpedia.org/resource/Brazil>\"],[\"Canada\",\"<http://db\\\npedia.org/resource/Canada>\"]]\nEnter text (all on one line)\n

\n\n

Note that entities that are not recognized as Wikipedia objects don’t get recognized.

Bag of Words Classification Model

The file Categorize.hs contains a simple bag of words classification model. To prepare the classification models, I collected a large set of labelled text. Labels were “chemistry”, “computers”, etc. I ranked words based on how often they appeared in training texts for a classification category, normalized by how often they appeared in all training texts. This example uses two auto-generated and data-specific Haskell files, one for single words and the other for two adjacent word pairs:

\nCategory1Gram.hs\n
\nCategory2Gram.hs\n

In NLP work, single words are sometimes called 1grams and two word adjacent pairs are referred to as 2grams. Here is a small amount of data from Category1Gram.hs:

 1 module Category1Gram (**onegrams**) where\n 2 \n 3 import qualified Data.Map as M\n 4 \n 5 chemistry = M.fromList [(\"chemical\", 1.15), (\"atoms\", 6.95),\n 6                         (\"reaction\", 6.7), (\"energy\", 6.05),\n 7                           ... ]\n 8 computers = M.fromList [(\"software\", 4.6), (\"network\", 4.65),\n 9                         (\"linux\", 3.6), (\"device\", 3.55), (\"computers\", 3.05),\n10                         (\"storage\", 2.7), (\"disk\", 2.3),\n11                           ... ]\n12 etc.                          \n

\n\n

Here is a small amount of data from Category2Gram.hs:

 1 module Category2Gram (**twograms**) where\n 2 \n 3 import qualified Data.Map as M\n 4 \n 5 chemistry = M.fromList [(\"chemical reaction\", 1.55),\n 6                         (\"atoms molecules\", 0.6), \n 7                         (\"periodic table\", 0.5),\n 8                         (\"chemical reactions\", 0.5),\n 9                         (\"carbon atom\", 0.5),\n10                          ... ]\n11 computers = M.fromList [(\"computer system\", 0.9),\n12                         (\"operating system\", 0.75),\n13                         (\"random memory\", 0.65),\n14                         (\"computer science\", 0.65),\n15                         (\"computer program\", 0.6),\n16                          ... ]\n17 etc.\n

\n\n

It is very common to use term frequencies for single words for classification models. One problem with using single words is that the evidence that any word gives for a classification is independent of the surrounding words in text being evaluated. By also using word pairs (two word combinations are often called 2grams or two-grams) we pick up patterns like “not good” giving evidence for negative sentiment even with the word “good” in text being evaluated. For my own work, I have a huge corpus of 1gram, 2gram, 3gram, and 4gram data sets. For the purposes of the following example program, I am only using 1gram and 2gram data.

The following listing shows the file Categorize.hs. Before looking at the entire example, let’s focus in on some of the functions I have defined for using the word frequency data to categorized text.

*Categorize> :t stemWordsInString\nstemWordsInString :: String -> [Char]\n*Categorize> stemWordsInString \"Banking industry is sometimes known for fraud.\"\n\"bank industri is sometim known for fraud\"\n

\n\n

stemScoredWordList is used to create a 1gram to word relevance score for each category. The keys are word stems.

*Categorize> stemScoredWordList onegrams \n[(\"chemistri\",fromList [(\"acid\",1.15),(\"acids\",0.8),(\"alcohol\",0.95),(\"atom\",4.45)\n

\n\n

Notice that “chemistri” is the stemmed version of “chemistry”, “bank” for “banks”, etc. stem2 is a 2gram frequency score by category mapping where the keys are word stems:

*Categorize> stem2\n[(\"chemistry\",fromList [(\"atom molecul\",0.6),(\"carbon atom\",0.5),(\"carbon carbon\",0.\\\n5),\n

\n\n

stem1 is like stem2, but for stemmed 1grams, not 2grams:

*Categorize> stem1\n[(\"chemistry\",fromList [(\"acid\",0.8),(\"chang\",1.05),(\"charg\",0.95),(\"chemic\",1.15),(\\\n\"chemistri\",1.45),\n

\n\n

score is called with a list or words and a word value mapping. Here is an example:

*Categorize> :t score\nscore\n  :: (Enum t, Fractional a, Num t, Ord a, Ord k) =>\n     [k] -> [(t1, M.Map k a)] -> [(t, a)]\n*Categorize> score [\"atom\", \"molecule\"] onegrams \n[(0,8.2),(25,2.4)]\n

\n\n

This output is more than a little opaque. The pair (0, 8.2) means that the input words [“atom”, “molecule”] have a score of 8.2 for category indexed at 0 and the pair (25,2.4) means that the input words have a score of 2.4 for the category at index 25. The category at index 0 is chemistry and the category at index 25 is physics as we can see by using the higher level function bestCategories1 that caluculates categories for a word sequence using 1gram word data:

*Categorize> :t bestCategories1\nbestCategories1 :: [[Char]] -> [([Char], Double)]\n*Categorize> bestCategories1 [\"atom\", \"molecule\"]\n[(\"chemistry\",8.2),(\"physics\",2.4)]\n

\n\n

The top level function bestCategories uses 1gram data. Here is an example for using it:

*Categorize> splitWords \"The chemist made a periodic table and explained a chemical \\\nreaction\"\n[\"the\",\"chemist\",\"made\",\"a\",\"periodic\",\"table\",\"and\",\"explained\",\"a\",\"chemical\",\"rea\\\nction\"]\n*Categorize> bestCategories1 $ splitWords \"The chemist made a periodic table and exp\\\nlained a chemical reaction\"\n[(\"chemistry\",11.25),(\"health_nutrition\",1.2)]\n

\n\n

Notice that these words were also classified as category “health_nutrition” but with a low score of 1.2. The score for “chemistry” is almost an order of magnitude larger. bestCategories sorts return values in “best first” order.

splitWords is used to split a string into word tokens before calling bestCategories.

Here is the entire example in file Categorize.hs:

 1 module Categorize (bestCategories, splitWords, bigram) where\n 2 \n 3 import qualified Data.Map as M\n 4 import Data.List (sortBy)\n 5 \n 6 import Category1Gram (onegrams)\n 7 import Category2Gram (twograms)\n 8 \n 9 import Sentence (segment)\n10 \n11 import Stemmer (stem)\n12 \n13 import Utils (splitWords, bigram, bigram_s)\n14 \n15 catnames1 = map fst onegrams\n16 catnames2 = map fst twograms\n17 \n18 stemWordsInString s = init $ concatMap ((++ \" \") . stem) (splitWords s)\n19 \n20 stemScoredWordList = map (\\(str,score) -> (stemWordsInString str, score))\n21 \n22 stem2 = map (\\(category, swl) ->\n23               (category, M.fromList (stemScoredWordList (M.toList swl))))\n24             twograms\n25 \n26 stem1 = map (\\(category, swl) ->\n27               (category, M.fromList (stemScoredWordList (M.toList swl))))\n28             onegrams\n29 \n30 scoreCat wrds amap =\n31   sum $ map (\\x ->  M.findWithDefault 0.0 x amap) wrds\n32 \n33 score wrds amap =\n34  filter (\\(a, b) -> b > 0.9) $ zip [0..] $ map (\\(s, m) -> scoreCat wrds m) amap\n35  \n36 cmpScore (a1, b1) (a2, b2) = compare b2 b1\n37                               \n38 bestCategoriesHelper wrds ngramMap categoryNames=\n39   let tg = bigram_s wrds in\n40     map (first (categoryNames !!)) $ sortBy cmpScore $ score wrds ngramMap\n41        \n42 bestCategories1 wrds =\n43   take 3 $ bestCategoriesHelper wrds onegrams catnames1\n44 \n45 bestCategories2 wrds =\n46   take 3 $ bestCategoriesHelper (bigram_s wrds) twograms catnames2\n47        \n48 bestCategories1stem wrds =\n49   take 3 $ bestCategoriesHelper wrds stem1 catnames1\n50 \n51 bestCategories2stem wrds =\n52   take 3 $ bestCategoriesHelper (bigram_s wrds) stem2 catnames2\n53 \n54 bestCategories :: [String] -> [(String, Double)]\n55 bestCategories wrds =\n56   let sum1 = M.unionWith (+) (M.fromList $ bestCategories1 wrds) ( M.fromList $ best\\\n57 Categories2 wrds)\n58       sum2 = M.unionWith (+) (M.fromList $ bestCategories1stem wrds) ( M.fromList $ \\\n59 bestCategories2stem wrds) \n60   in sortBy cmpScore $ M.toList $ M.unionWith (+) sum1 sum2\n61       \n62 main = do\n63     let s = \"The sport of hocky is about 100 years old by ahdi dates. American Footb\\\n64 all is a newer sport. Programming is fun. Congress passed a new budget that might he\n65 lp the economy. The frontier initially was a value path. The ai research of john mcc\n66 arthy.\"\n67     print $ bestCategories1 (splitWords s)    \n68     print $ bestCategories1stem (splitWords s)\n69     print $ score (splitWords s) onegrams\n70     print $ score (bigram_s (splitWords s)) twograms\n71     print $ bestCategories2 (splitWords s)\n72     print $ bestCategories2stem (splitWords s)\n73     print $ bestCategories (splitWords s)\n

\n\n

Here is the output:

 1 $ stack ghci\n 2 :l Categorize.hs\n 3 *Categorize> main\n 4 [(\"computers_ai\",17.900000000000002),(\"sports\",9.75),(\"computers_ai_search\",6.2)]\n 5 [(\"computers_ai\",18.700000000000003),(\"computers_ai_search\",8.1),(\"computers_ai_lear\\\n 6 ning\",5.7)]\n 7 [(2,17.900000000000002),(3,1.75),(4,5.05),(6,6.2),(9,1.1),(10,1.2),(21,2.7),(26,1.1)\\\n 8 ,(28,1.6),(32,9.75)]\n 9 [(2,2.55),(6,1.0),(32,2.2)]\n10 [(\"computers_ai\",2.55),(\"sports\",2.2),(\"computers_ai_search\",1.0)]\n11 [(\"computers_ai\",1.6)]\n12 [(\"computers_ai\",40.75000000000001),(\"computers_ai_search\",15.3),(\"sports\",11.95),(\"\\\n13 computers_ai_learning\",5.7)]\n

\n\n

Given that the variable s contains some test text, line 4 of this output was generated by evaluating bestCategories1 (splitWords s), lines 5-6 by evaluating bestCategories1stem (splitWords s), lines 7-8 from score (splitWords s) onegrams, line 9 from core (bigram_s (splitWords s)) twograms, line 10 from bestCategories2 (splitWords s), line 11 from bestCategories2stem (splitWords s), and lines 12-13 from bestCategories (splitWords s).

I called all of the utility fucntions in function main to demonstrate what they do but in practice I just call function bestCategories in my applications.

Text Summarization

This application uses both the Categorize.hs code and the 1gram data from the last section. The algorithm I devised for this example is based on a simple idea: we categorize text and keep track of which words provide the strongest evidence for the highest ranked categories. We then return a few sentences from the original text that contain the largest numbers of these important words.

module Summarize (summarize, summarizeS) where\n\nimport qualified Data.Map as M\nimport Data.List.Utils (replace)\nimport Data.Maybe (fromMaybe)\n\nimport Categorize (bestCategories)\nimport Sentence (segment)\nimport Utils (splitWords, bigram_s, cleanText)\n\nimport Category1Gram (onegrams)\nimport Category2Gram (twograms)\n\nscoreSentenceHelper words scoreMap = -- just use 1grams for now\n  sum $ map (\\word ->  M.findWithDefault 0.0 word scoreMap) words\n\nsafeLookup key alist =\n  fromMaybe 0 $ lookup key alist\n \nscoreSentenceByBestCategories words catDataMaps bestCategories =\n  map (\\(category, aMap) -> \n        (category, safeLookup category bestCategories * \n                   scoreSentenceHelper words aMap)) catDataMaps\n\nscoreForSentence words catDataMaps bestCategories =  \n  sum $ map snd $ scoreSentenceByBestCategories words catDataMaps bestCategories\n\nsummarize s =\n  let words = splitWords $ cleanText s\n      bestCats = bestCategories words\n      sentences = segment s\n      result1grams = map (\\sentence ->\n                           (sentence,\n                            scoreForSentence (splitWords sentence)\n                                             onegrams bestCats)) \n                         sentences\n      result2grams = map (\\sentence ->\n                           (sentence,\n                            scoreForSentence (bigram_s (splitWords sentence))\n                                             twograms bestCats)) \n                         sentences\n      mergedResults = M.toList $ M.unionWith (+)\n                      (M.fromList result1grams) (M.fromList result1grams)\n      c400 = filter (\\(sentence, score) -> score > 400) mergedResults\n      c300 = filter (\\(sentence, score) -> score > 300) mergedResults\n      c200 = filter (\\(sentence, score) -> score > 200) mergedResults\n      c100 = filter (\\(sentence, score) -> score > 100) mergedResults\n      c000 = mergedResults in\n  if not (null c400) then c400 else if not (null c300) then c300 else if not (null c\\\n200) then c200 else if not (null c100) then c100 else c000\n\nsummarizeS s =\n  let a = replace \"\\\"\" \"'\" $ concatMap (\\x -> fst x ++ \" \") $ summarize s in\n  if not (null a) then a else safeFirst $ segment s where\n    safeFirst x \n      | length x > 1 = head x ++ x !! 1\n      | not (null x)   = head x\n      | otherwise    = \"\"\n      \nmain = do     \n  let s = \"Plunging European stocks, wobbly bonds and grave concerns about the healt\\\nh of Portuguese lender Banco Espirito Santo SA made last week feel like a rerun of t\nhe euro crisis, but most investors say it was no more than a blip for a resurgent re\ngion. Banco Espirito Santo has been in investors’ sights since December, when The Wa\nll Street Journal first reported on accounting irregularities at the complex firm. N\nerves frayed on Thursday when Banco Espirito Santo's parent company said it wouldn't\n be able to meet some short-term debt obligations.\"\n  print $ summarize s\n  print $ summarizeS s\n

\n\n

Lazy evaluation allows us in function summarize to define summaries of various numbers of sentences, but not all of these possible summaries are calculated.

$ stack ghci\n*Main ... > :l Summarize.hs\n*Summarize> main\n[(\"Nerves frayed on Thursday when Banco Espirito Santo's parent company said it woul\\\ndn't be able to meet some short-term debt obligations.\",193.54500000000002)]\n\"Nerves frayed on Thursday when Banco Espirito Santo's parent company said it wouldn\\\n't be able to meet some short-term debt obligations. \"\n

\n\n

Part of Speech Tagging

We close out this chapter with the Haskell version of my part of speech (POS) tagger that I originally wrote in Common Lisp, then converted to Ruby and Java. The file LexiconData.hs is similar to the lexical data files seen earlier: I am defining a map where keys a words and map values are POS tokens like NNP (proper noun), RB (adverb), etc. The file README.md contains a complete list of POS tag definitions.

The example code and data for this section is in the directory FastTag.

This listing shows a tiny representative part of the POS definitions in LexiconData.hs:

lexicon = M.fromList [(\"AARP\", \"NNP\"), (\"Clinic\", \"NNP\"), (\"Closed\", \"VBN\"),\n                      (\"Robert\", \"NNP\"), (\"West-German\", \"JJ\"),\n                      (\"afterwards\", \"RB\"), (\"arises\", \"VBZ\"),\n                      (\"attacked\", \"VBN\"), ...]\n

\n\n

Before looking at the code example listing, let’s see how the functions defined in fasttag.hs work in a GHCi repl:

*Main LexiconData> bigram [\"the\", \"dog\", \"ran\",\n                           \"around\", \"the\", \"tree\"]\n[[\"the\",\"dog\"],[\"dog\",\"ran\"],[\"ran\",\"around\"],\n [\"around\",\"the\"],[\"the\",\"tree\"]]\n*Main LexiconData> tagHelper \"car\"\n[\"car\",\"NN\"]\n*Main LexiconData> tagHelper \"run\"\n[\"run\",\"VB\"]\n*Main LexiconData> substitute [\"the\", \"dog\", \"ran\", \"around\",\n                               \"the\", \"tree\"]\n[[[\"the\",\"DT\"],[\"dog\",\"NN\"]],[[\"dog\",\"NN\"],[\"ran\",\"VBD\"]],\n [[\"ran\",\"VBD\"],[\"around\",\"IN\"]],[[\"around\",\"IN\"],[\"the\",\"DT\"]],\n [[\"the\",\"DT\"],[\"tree\",\"NN\"]]]\n*Main LexiconData> fixTags $  substitute [\"the\", \"dog\", \"ran\", \n                                          \"around\", \"the\", \"tree\"]\n[\"NN\",\"VBD\",\"IN\",\"DT\",\"NN\"]\n

\n\n

Function bigram takes a list or words and returns a list of word pairs. We need the word pairs because parts of the tagging algorithm needs to see a word with its preceding word. In an imperative language, I would loop over the words and for a word at index i I would have the word at index i - 1. In a functional language, we avoid using loops and in this case create a list of adjacent word pairs to avoid having to use an explicit loop. I like this style of functional programming but if you come from years of using imperative language like Java and C++ it takes some getting used to.

tagHelper converts a word into a list of the word and its likely tag. substitute applies tagHelper to a list of words, getting the most probable tag for each word. The function fixTags will occasionally override the default word tags based on a few rules that are derived from Eric Brill’s paper A Simple Rule-Based Part of Speech Tagger.

Here is the entire example:

 1 module Main where\n 2 \n 3 import qualified Data.Map as M\n 4 import Data.Strings (strEndsWith, strStartsWith)\n 5 import Data.List (isInfixOf)\n 6 \n 7 import LexiconData (lexicon)\n 8 \n 9 bigram :: [a] -> [[a]]\n10 bigram [] = []\n11 bigram [_] = []\n12 bigram xs = take 2 xs : bigram (tail xs)\n13 \n14 containsString word substring = isInfixOf substring word\n15 \n16 fixTags twogramList =\n17   map\n18   -- in the following inner function, [last,current] might be bound,\n19   -- for example, to [[\"dog\",\"NN\"],[\"ran\",\"VBD\"]]\n20   (\\[last, current] ->\n21     -- rule 1: DT, {VBD | VBP} --> DT, NN\n22     if last !! 1 == \"DT\" && (current !! 1 == \"VBD\" ||\n23                              current !! 1 == \"VB\" ||\n24                              current !! 1 == \"VBP\")\n25     then \"NN\" \n26     else\n27       -- rule 2: convert a noun to a number (CD) if \".\" appears in the word\n28       if (current !! 1) !! 0 == 'N' && containsString (current !! 0) \".\"\n29       then \"CD\"\n30       else\n31         -- rule 3: convert a noun to a past participle if\n32         --         words.get(i) ends with \"ed\"\n33         if (current !! 1) !! 0 == 'N' && strEndsWith (current !! 0) \"ed\"\n34         then \"VBN\"\n35         else\n36           -- rule 4: convert any type to adverb if it ends in \"ly\"\n37           if strEndsWith (current !! 0) \"ly\"\n38           then \"RB\"\n39           else\n40             -- rule 5: convert a common noun (NN or NNS) to an\n41             --         adjective if it ends with \"al\"\n42             if strStartsWith (current !! 1) \"NN\" &&\n43                strEndsWith (current !! 1) \"al\"\n44             then \"JJ\"\n45             else\n46               -- rule 6: convert a noun to a verb if the preceeding\n47               --         word is \"would\"\n48               if strStartsWith (current !! 1) \"NN\" &&\n49                  (last !! 0) == \"would\" -- should be case insensitive\n50               then \"VB\"\n51               else\n52                 -- rule 7: if a word has been categorized as a\n53                 --         common noun and it ends with \"s\",\n54                 --         then set its type to plural common noun (NNS)\n55                 if strStartsWith (current !! 1) \"NN\" &&\n56                    strEndsWith (current !! 0) \"s\"\n57                 then \"NNS\"\n58                 else\n59                   -- rule 8: convert a common noun to a present\n60                   --         participle verb (i.e., a gerand)\n61                   if strStartsWith (current !! 1) \"NN\" &&\n62                      strEndsWith (current !! 0) \"ing\"\n63                   then \"VBG\"\n64                   else (current !! 1))\n65  twogramList\n66   \n67 substitute tks = bigram $ map tagHelper tks\n68 \n69 tagHelper token =\n70   let tags = M.findWithDefault [] token lexicon in\n71   if tags == [] then [token, \"NN\"] else [token, tags]\n72 \n73 tag tokens = fixTags $ substitute ([\"\"] ++ tokens)\n74 \n75 \n76 main = do\n77   let tokens = [\"the\", \"dog\", \"ran\", \"around\", \"the\", \"tree\", \"while\",\n78                 \"the\", \"cat\", \"snaked\", \"around\", \"the\", \"trunk\",\n79                 \"while\", \"banking\", \"to\", \"the\", \"left\"]\n80   print $ tag tokens\n81   print $ zip tokens $ tag tokens\n

\n\n

*Main LexiconData> main\n[\"DT\",\"NN\",\"VBD\",\"IN\",\"DT\",\"NN\",\"IN\",\"DT\",\"NN\",\"VBD\",\"IN\",\"DT\",\n \"NN\",\"IN\",\"VBG\",\"TO\",\"DT\",\"VBN\"]\n[(\"the\",\"DT\"),(\"dog\",\"NN\"),(\"ran\",\"VBD\"),(\"around\",\"IN\"),\n (\"the\",\"DT\"),(\"tree\",\"NN\"),(\"while\",\"IN\"),(\"the\",\"DT\"),\n (\"cat\",\"NN\"),(\"snaked\",\"VBD\"),(\"around\",\"IN\"),(\"the\",\"DT\"),\n (\"trunk\",\"NN\"),(\"while\",\"IN\"),(\"banking\",\"VBG\"),(\"to\",\"TO\"),\n (\"the\",\"DT\"),(\"left\",\"VBN\")]\n

\n\n

The README.md file contains definitions of the POS definitions. Here are the ones used in this example:

DT Determiner               the,some\nNN noun                     dog,cat,road\nVBD verb, past tense        ate,ran\nIN Preposition              of,in,by\n

\n\n

Natural Language Processing Wrap Up

NLP is a large topic. I have attempted to show you just the few tricks that I use often and are simple to implement. I hope that you reuse the code in this chapter in your own projects when you need to detect entities, classify text, summarize text, and assign part of speech tags to words in text.

Linked Data and the Semantic Web

I am going to show you how to query semantic web data sources on the web and provide examples for how you might use this data in applications. I have written two previous books on the semantic web, one covering Common Lisp and the other covering JVM languages Java, Scala, Clojure, and Ruby. You can read these recent eBooks online for free on my Leanpub author’s page. If you enjoy the light introduction in this chapter then please do read my other eBooks that cover in more detail semantic web material on RDF, RDFS, and SPARQL.

I like to think of the semantic web and linked data resources as:

\nA source of structured data on the web. These resources are called SPARQL endpoints.\n
\nData is represented by data triples: subject, predicate, and object. The subject of one triple can be the object of another triple. Predicates are relationships; a few examples: “owns”, “is part of”, “author of”, etc.\n
\nData that is accessed via the SPARQL query language.\n
\nA source of data that may or may not be available. SPARQL endpoints are typically available for free use and they are sometimes unavailable. Although not covered here, I sometimes work around this problem by adding a caching layer to SPARQL queries (access key being a SPARQL query string, the value being the query results). This caching speeds up development and running unit tests, and sometimes saves a customer demo when a required SPARQL endpoint goes offline at an inconvenient time.\n

DBPedia is the semantic web version of Wikipedia. The many millions of data triples that make up DBPedia are mostly derived from the structured “info boxes” on Wikipedia pages.

As you are learning SPARQL use the DBPedia SPARQL endpoint to practice. As a practitioner who uses linked data, for any new project I start by identifying SPARQL endpoints for possibly useful data. I then interactively experiment with SPARQL queries to extract the data I need. Only when I am satisfied with the choice of SPARQL endpoints and SPARQL queries do I write any code to automatically fetch linked data for my application.

Pro tip: I mentioned SPARQL query caching. I sometimes cache query results in a local database, saving the returned RDF data indexed by the SPARQL query. You can also store the cache timestamp and refresh the cache every few weeks as needed. In addition to making development and unit testing faster, your applications will be more resilient.

In the last chapter “Natural Language Processing Tools” we resolved entities in natural language text to DBPedia (semantic web SPAQL endpoint for Wikipedia) URIs. Here we will use some of these URIs to demonstrate fetching real world knowledge that you might want to use in applications.

The SPARQL Query Language

Example RDF N3 triples (subject, predicate, object) might look like:

<http://www.markwatson.com>\n  <http://dbpedia.org/ontology/owner>\n  \"Mark Watson\" .\n

\n\n

Element of triples can be URIs or string constants. Triples are often written all on one line; I split it to three lines to fit the page width. Here the subject is the URI for my web site, the predicate is a URI defining an ownership relationship, and the object is a string literal.

If you want to see details for any property or other URI you see, then “follow your nose” and open the URI in a web browser. For example remove the brackets from the owner property URI http://dbpedia.org/ontology/owner and open it in a web browser. For working with RDF data programmatically, it is convenient using full URI. For humans reading RDF, the N3 notation is better because it supports defining URI standard prefixes for use as abbreviations; for example:

prefix ontology: <http://dbpedia.org/ontology/>\n\n<http://www.markwatson.com>\n  ontology:owner\n  \"Mark Watson\" .\n

\n\n

If you wanted to find all things that I own (assuming this data was in a public RDF repository, which it isn’t) then we might think to match the pattern:

prefix ontology: <http://dbpedia.org/ontology/>\n\n?subject ontology:owner \"Mark Watson\"\n

\n\n

And return all URIs matching the variable ?subject as the query result. This is the basic idea of making SPARQL queries.

The following SPARQL query will be implemented later in Haskell using the HSparql library:

prefix resource: <http://dbpedia.org/resource/>\nprefix dbpprop: <http://dbpedia.org/property/>\nprefix foaf: <http://xmlns.com/foaf/0.1/>\n\nSELECT *\nWHERE {\n    ?s dbpprop:genre resource:Web_browser .\n    ?s foaf:name ?name .\n} LIMIT 5\n

\n\n

In this last SPARQL query example, the triple patterns we are trying to match are inside a WHERE clause. Notice that in the two triple patterns, the subject field of each is the variable ?s. The first pattern matches all DBPedia triples with a predicate http://dbpedia.org/property/genre and an object equal to http://dbpedia.org/resource/Web_browser. We then find all triples with the same subject but with a predicate equal to http://xmlns.com/foaf/0.1/name.

Each result from this query will contain two values for variables ?s and ?name: a DBPedia URI for some thing and the name for that thing. Later we will run this query using Haskell code and you can see what the output might look like.

Sometimes when I am using a specific SPARQL query in an application, I don’t bother defining prefixes and just use URIs in the query. As an example, suppose I want to return the Wikipedia (or DBPedia) abstract for IBM. I might use a query such as:

select * where {\n  <http://dbpedia.org/resource/IBM>\n  <http://dbpedia.org/ontology/abstract>\n  ?o .\n  FILTER langMatches(lang(?o), \"EN\")\n} LIMIT 100\n

\n\n

If you try this query using the web interface for DBPedia SPARQL queries you get just one result because of the FILTER option that only returns English language results. You could also use FR for French results, GE for German results, etc.

A Haskell HTTP Based SPARQL Client

One approach to query the DBPedia SPARQL endpoint is to build a HTTP GET request, send it to the SPARQL endpoint server, and parse the returned XML response. We will start with this simple approach. You will recognize the SPARQL query from the last section:

 1 {-# LANGUAGE OverloadedStrings #-}\n 2 \n 3 module HttpSparqlClient where\n 4 \n 5 import Network.HTTP.Conduit (simpleHttp)\n 6 import Network.HTTP.Base (urlEncode)\n 7 import Text.XML.HXT.Core\n 8 import Text.HandsomeSoup\n 9 import qualified Data.ByteString.Lazy.Char8 as B\n10 \n11 buildQuery :: String -> [Char]\n12 buildQuery sparqlString =\n13   \"http://dbpedia.org/sparql/?query=\" ++ urlEncode sparqlString\n14   \n15 main :: IO ()\n16 main = do\n17   let query = buildQuery \"select * where {<http://dbpedia.org/resource/IBM> <http://\\\n18 dbpedia.org/ontology/abstract> ?o . FILTER langMatches(lang(?o), \\\"EN\\\")} LIMIT 100\"\n19   res <- simpleHttp query\n20   let doc = readString []  (B.unpack res)\n21   putStrLn \"\\nAbstracts:\\n\"\n22   abstracts <- runX $ doc >>> css \"binding\" >>>\n23                               (getAttrValue \"name\" &&& (deep getText))\n24   print abstracts\n

\n\n

The function buildQuery defined in lined 11-13 takes any SPARQL query, URL encodes it so it can be passed as part of a URI, and builds a query string for the DBPedia SPARQL endpoint. The returned data is in XML format. In lines 22-23 I am using the XHT parsing library to extract the names (values bound to the variable ?o in the query in line 17). I covered the use of the HandsomeSoup parsing library in the chapter Web Scraping.

We use runX to execute a series of operations on an XML document (the doc variable). We first select all elements in doc that have the CSS class binding using the css function. Next we extract the value of the name attribute from each selected element using getAttrValue and also extract the text inside the element using the function deep.\nThe &&& operator is used to combine the two values for the name attribute and the element text into a tuple.

In the main function, we use the utility function simpleHttp in line 19 to fetch the results as a ByteString and in line 20 we unpack this to a regular Haskell String.

 1 Prelude> :l HttpSparqlClient.hs \n 2 [1 of 1] Compiling HttpSparqlClient ( HttpSparqlClient.hs, interpreted )\n 3 Ok, modules loaded: HttpSparqlClient.\n 4 *HttpSparqlClient> main\n 5 \n 6 Abstracts:\n 7 \n 8 [(\"o\",\"International Business Machines Corporation (commonly referred to as IBM) is \\\n 9 an American multinational technology and consulting corporation, with corporate head\n10 quarters in Armonk, New York.\n11   ...)]\n

\n\n

On line 8, “o” corresponds to the SPARQL query variable ?o and the value bound to the variable ?o is the string “International Business…”.

Querying Remote SPARQL Endpoints

We will write some code in this section to make the example query to get the names of web browsers from DBPedia. In the last section we made a SPARQL query using fairly low level Haskell libraries. We will be using the high level library HSparql to build SPARQL queries and call the DBPedia SPARQL endpoint.

The example in this section can be found in SparqlClient/TestSparqlClient.hs. In the main function notice how I have commented out printouts of the raw query results. Because Haskell is type safe, extracting the values wrapped in query results requires knowing RDF element return types. I will explain this matching after the program listing:

 1 -- simple experiments with the excellent HSparql library\n 2 \n 3 module Main where\n 4 \n 5 import Database.HSparql.Connection (BindingValue(Bound))\n 6 \n 7 import Data.RDF hiding (triple)\n 8 import Database.HSparql.QueryGenerator\n 9 import Database.HSparql.Connection (selectQuery)\n10     \n11 webBrowserSelect :: Query SelectQuery\n12 webBrowserSelect = do\n13     resource <- prefix \"dbprop\" (iriRef \"http://dbpedia.org/resource/\")\n14     dbpprop  <- prefix \"dbpedia\" (iriRef \"http://dbpedia.org/property/\")\n15     foaf     <- prefix \"foaf\" (iriRef \"http://xmlns.com/foaf/0.1/\")\n16     x    <- var\n17     name <- var\n18     triple x (dbpprop .:. \"genre\") (resource .:. \"Web_browser\")\n19     triple x (foaf .:. \"name\") name\n20 \n21     return SelectQuery { queryVars = [name] }\n22 \n23 companyAbstractSelect :: Query SelectQuery\n24 companyAbstractSelect = do\n25     resource <- prefix \"dbprop\" (iriRef \"http://dbpedia.org/resource/\")\n26     ontology <- prefix \"ontology\" (iriRef \"http://dbpedia.org/ontology/\")\n27     o <- var\n28     triple (resource .:. \"Edinburgh_University_Press\") (ontology .:. \"abstract\") o\n29     return SelectQuery { queryVars = [o] }\n30 \n31 companyTypeSelect :: Query SelectQuery\n32 companyTypeSelect = do\n33     resource <- prefix \"dbprop\" (iriRef \"http://dbpedia.org/resource/\")\n34     ontology <- prefix \"ontology\" (iriRef \"http://dbpedia.org/ontology/\")\n35     o <- var\n36     triple (resource .:. \"Edinburgh_University_Press\") (ontology .:. \"type\") o\n37     return SelectQuery { queryVars = [o] }\n38 \n39 main :: IO ()\n40 main = do\n41   sq1 <- selectQuery \"http://dbpedia.org/sparql\" companyAbstractSelect\n42   --putStrLn \"\\nRaw results of company abstract SPARQL query:\\n\"\n43   --print sq1\n44   putStrLn \"\\nWeb browser names extracted from the company abstract query results:\\n\"\n45   case sq1 of\n46     Just a -> print $ map (\\[Bound (LNode (PlainLL s _))] -> s) a\n47     Nothing -> putStrLn \"nothing\"\n48   sq2 <- selectQuery \"http://dbpedia.org/sparql\" companyTypeSelect\n49   --putStrLn \"\\nRaw results of company type SPARQL query:\\n\"\n50   --print sq2\n51   putStrLn \"\\nWeb browser names extracted from the company type query results:\\n\"\n52   case sq2 of\n53     Just a -> print $ map (\\[Bound (UNode  s)] -> s) a\n54     Nothing -> putStrLn \"nothing\"\n55   sq3 <- selectQuery \"http://dbpedia.org/sparql\" webBrowserSelect\n56   --putStrLn \"\\nRaw results of SPARQL query:\\n\"\n57   --print sq3\n58   putStrLn \"\\nWeb browser names extracted from the query results:\\n\"\n59   case sq3 of\n60     Just a -> print $ map (\\[Bound (LNode (PlainLL s _))] -> s) a\n61     Nothing -> putStrLn \"nothing\"\n

\n\n

Haskell Code for SPARQL Queries with HSparql

This provided Haskell code demonstrates the use of the HSparql library to interact with a SPARQL endpoint (specifically, DBpedia) to perform semantic queries on linked data.

SPARQL Query Definitions

It begins by defining three SPARQL queries, each constructed using the Query monad provided by HSparql. These queries are:

\n
\n
webBrowserSelect:
\n
\n
\n
- \nThis query aims to retrieve the names of entities categorized as web browsers.\n
- \nIt utilizes prefixes to simplify the representation of URIs within the query.\n
- \nIt selects entities (x) that have a “genre” property linking them to the concept of a “Web_browser” and then retrieves their “name.”\n
\n
\n
\n
\n
companyAbstractSelect:
\n
\n
\n
- \nThis query targets information about the “Edinburgh University Press.”\n
- \nIt seeks to retrieve the “abstract” associated with this entity, which provides a concise summary or description.\n
\n
\n
\n
\n
companyTypeSelect:
\n
\n
\n
- \nSimilar to the previous query, this one focuses on the “Edinburgh University Press” but retrieves its “type,” which indicates the category or class it belongs to within the DBpedia ontology.\n
\n
\n

\n`main` Function

The main function serves as the entry point of the program. It performs the following actions:

\n
Query Execution: It executes each of the defined SPARQL queries against the DBpedia SPARQL endpoint using the selectQuery function. This function returns the query results wrapped in a Maybe type to handle potential query failures.
\n
\n
Result Processing: The code then pattern matches on the query results to extract and process the relevant information. It handles both successful query results (Just a) and potential query failures (Nothing).
\n
\n
Output: Finally, the extracted information (primarily names in this case) is printed to the console, providing the user with the desired results of the SPARQL queries.
\n

Summary

In summary, this Haskell code showcases a practical example of how to leverage the HSparql library to interact with a SPARQL endpoint (DBpedia) to retrieve and process structured data from the Semantic Web. It demonstrates the construction of SPARQL queries, their execution, and the subsequent handling and presentation of query results.

Notes on matching result types of query results:

You will notice how I have commented out print statements in the last example. When trying new queries you need to print out the results in order to know how to extract the wrapped query results. Let’s look at a few examples:

If we print the value for sq1:

Raw results of company abstract SPARQL query:\n\nJust [[Bound (LNode (PlainLL \"Edinburgh University Press ...\n

\n\n

we see that inside a Just we have a list of lists. Each inner list is a Bound wrapping types defined in HSparql. We would unwrap sq1 using:

  case sq1 of\n    Just a -> print $ map (\\[Bound (LNode (PlainLL s _))] -> s) a\n    Nothing -> putStrLn \"nothing\"\n

\n\n

In a similar way I printed out the values of sq2 and sq3 to see the form os case statement I would need to unwrap them.

The output from this example with three queries to the DBPedia SPARQL endpoint is:

Web browser names extracted from the company abstract query results in sq1:\n\n[\"Edinburgh University Press \\195\\168 una casa editrice scientifica di libri accadem\\\nici e riviste, con sede a Edimburgo, in Scozia.\",\"Edinburgh University Press \\195\\16\n9 uma editora universit\\195\\161ria com base em Edinburgh, Esc\\195\\179cia.\",\"Edinburg\nh University Press is a scholarly publisher of academic books and journals, based in\n Edinburgh, Scotland.\"]\n\nThe type of company is extracted from the company type query results in sq2:\n\n[\"http://dbpedia.org/resource/Publishing\"]\n\nWeb browser names extracted from the query results in sq3:\n\n[\"Grail\",\"ViolaWWW\",\"Kirix Strata\",\"SharkWire Online\",\"MacWeb\",\"Camino\",\"eww\",\"TenFo\\\nurFox\",\"WiseStamp\",\"X-Smiles\",\"Netscape Navigator 2\",\"SimpleTest\",\"AWeb\",\"IBrowse\",\"\niCab\",\"ANT Fresco\",\"Netscape Navigator 9.0\",\"HtmlUnit\",\"ZAC Browser\",\"ELinks\",\"ANT G\nalio\",\"Nintendo DSi Browser\",\"Nintendo DS Browser\",\"Netscape Navigator\",\"NetPositive\n\",\"OmniWeb\",\"Abaco\",\"Flock\",\"Steel\",\"Kazehakase\",\"GNU IceCat\",\"FreeWRL\",\"UltraBrowse\nr\",\"AMosaic\",\"NetCaptor\",\"NetSurf\",\"Netscape Browser\",\"SlipKnot\",\"ColorZilla\",\"Inter\nnet Channel\",\"Obigo Browser\",\"Swiftfox\",\"BumperCar\",\"Swiftweasel\",\"Swiftdove\",\"IEs4L\ninux\",\"MacWWW\",\"IBM Lotus Symphony\",\"SlimBrowser\",\"cURL\",\"FoxyTunes\",\"Iceweasel\",\"Me\nnuBox\",\"Timberwolf web browser\",\"Classilla\",\"Rockmelt\",\"Galeon\",\"Links\",\"Netscape Na\nvigator\",\"NCSA Mosaic\",\"MidasWWW\",\"w3m\",\"PointerWare\",\"Pogo Browser\",\"Oregano\",\"Avan\nt Browser\",\"Wget\",\"NeoPlanet\",\"Voyager\",\"Amaya\",\"Midori\",\"Sleipnir\",\"Tor\",\"AOL Explo\nrer\"]\n

\n\n

Linked Data and Semantic Web Wrap Up

If you enjoyed the material on linked data and DBPedia then please do get a free copy of one of my semantic web books on my website book page as well as other SPARQL and linked data tutorials on the web.

Structured and semantically labelled data, when it is available, is much easier to process and use effectively than raw text and HTML collected from web sites.

Web Scraping

Note: the code for this example replaced November 9, 2024.

In my past work I usually used the Ruby and Python scripting languages for web scraping but as I use the Haskell language more often for projects both large and small I am now using Haskell for web scraping, data collection, and data cleaning tasks. If you worked through the tutorial chapter on impure Haskell programming then you already know most of what you need to understand this chapter. Here we will walk through a few short examples for common web scraping tasks.

Before we start a tutorial about web scraping I want to point out that much of the information on the web is copyright and the first thing that you should do is to read the terms of service for web sites to insure that your use of web scraped data conforms with the wishes of the persons or organizations who own the content and pay to run scraped web sites.

As we saw in the last chapter on linked data there is a huge amount of structured data available on the web via web services, semantic web/linked data markup, and APIs. That said, you will frequently find text (usually HTML) that is useful on web sites. However, this text is often at least partially unstructured and in a messy and frequently changing format because web pages are meant for human consumption and making them easy to parse and use by software agents is not a priority of web site owners.\nHere is t\nhe code for the entire example in directory haskell_tutorial_cookbook_examples/WebScraping (code description follows the listing):

{-# LANGUAGE OverloadedStrings #-}\n\nimport Network.HTTP.Simple\nimport Text.HTML.TagSoup\nimport Data.Text (Text)\nimport qualified Data.Text as T\nimport qualified Data.Text.IO as TIO\nimport qualified Data.ByteString.Lazy.Char8 as BL8\nimport Data.Maybe (mapMaybe)\n\nmain :: IO ()\nmain = do\n    -- Fetch the HTML content\n    response <- httpLBS \"https://markwatson.com/\"\n    let body = BL8.unpack $ getResponseBody response\n        tags = parseTags body\n\n    -- Extract and print headers\n    let headers = getResponseHeaders response\n    putStrLn \"Headers:\"\n    mapM_ print headers\n\n    -- Extract and print all text content\n    let texts = extractTexts tags\n    putStrLn \"\\nText Content:\"\n    TIO.putStrLn texts\n\n    -- Extract and print all links\n    let links = extractLinks tags\n    putStrLn \"\\nLinks:\"\n    mapM_ TIO.putStrLn links\n\n-- Function to extract all text content from tags\nextractTexts :: [Tag String] -> Text\nextractTexts = T.unwords . map (T.strip . T.pack) . filter (not . null) . mapMaybe m\\\naybeTagText\n\n-- Function to extract all links from tags\nextractLinks :: [Tag String] -> [Text]\nextractLinks = map (T.pack . fromAttrib \"href\") . filter isATag\n  where\n    isATag (TagOpen \"a\" _) = True\n    isATag _               = False\n

\n\n

This Haskell program retrieves and processes the content of the webpage at https://markwatson.com/. It utilizes the http-conduit library to perform an HTTP GET request, fetching the HTML content of the specified URL. The response body, initially in a lazy ByteString format, is converted to a String using BL8.unpack to facilitate subsequent parsing operations.

For parsing the HTML content, the program employs the TagSoup library, which is adept at handling both well-formed and malformed HTML. The parseTags function processes the HTML String into a list of Tag String elements, representing the structure of the HTML document. This parsed representation enables efficient extraction of specific components, such as headers, text content, and hyperlinks.

The program defines two functions, extractTexts and extractLinks, to extract text content and hyperlinks, respectively. The extractTexts function filters the parsed tags to identify text nodes, removes any empty strings, converts them to Text, strips leading and trailing whitespace, and concatenates them into a single Text value. The extractLinks function filters for anchor tags, extracts their href attributes, and converts these URLs to Text.

In the main function, after fetching and parsing the HTML content, the program retrieves and prints the HTTP response headers using getResponseHeaders. It then calls extractTexts to obtain and display the textual content of the webpage, followed by extractLinks to list all hyperlinks present in the HTML. This structured approach allows for a clear and organized extraction of information from the specified webpage.

Here is some example output (shortened for brevity):

 1  $ cabal run TagSoupTest\n 2 Headers:\n 3 (\"Date\",\"Sat, 09 Nov 2024 18:12:46 GMT\")\n 4 (\"Content-Type\",\"text/html; charset=utf-8\")\n 5 (\"Transfer-Encoding\",\"chunked\")\n 6 (\"Connection\",\"keep-alive\")\n 7 (\"Last-Modified\",\"Mon, 04 Nov 2024 22:52:48 GMT\")\n 8 (\"Access-Control-Allow-Origin\",\"*\")\n 9 \n10 Text Content:\n11      Mark Watson: AI Practitioner and Author of 20+ AI Books | Mark Watson          \\\n12        window.dataLayer = window.dataLayer || [];\n13        function gtag(){dataLayer.push(arguments);}\n14        gtag('js', new Date());\n15        gtag('config', 'G-MJNL6DY9ZQ');            Read My Blog on Blogspot      Read\\\n16  My Blog on Substack      Consulting      Fun stuff      My Books      Open Source  \n17    Privacy Policy       Mark Watson AI Practitioner and Consultant Specializing in L\n18 arge Language Models, LangChain/Llama-Index Integrations, Deep Learning, and the Sem\n19 antic Web   I am the author of 20+ books on Artificial Intelligence, Python, Common \n20 Lisp, Deep Learning, Haskell, Clojure, Java, Ruby, Hy language, and the Semantic Web\n21 . I have 55 US Patents.         My customer list includes: Google, Capital One, Baby\n22 list, Olive AI, CompassLabs, Mind AI, Disney, SAIC, Americast, PacBell, CastTV, Lutr\n23 is Technology, Arctan Group, Sitescout.com, Embed.ly, and Webmind Corporation.\n24 \n25 Links:\n26 https://mark-watson.blogspot.com/\n27 https://marklwatson.substack.com\n28 #consulting\n29 #fun\n30 #books\n31 #opensource\n32 https://markwatson.com/privacy.html\n

\n\n

Web Scraping Wrap Up

There are many Haskell library options for web scraping and cleaning data. In this chapter I showed you just what I use in my projects.

The material in this chapter and the chapters on text processing and linked data should be sufficient to get you started using online data sources in your applications.

Using Relational Databases

We will see how to use popular libraries for accessing the sqlite and Postgres (sometimes also called PostgeSQL) databases in this chapter. I assume that you are already familiar with SQL.

Database Access for Sqlite

We will use the sqlite-simple library in this section to access Sqlite databases and use the similar library postgresql-simple in the next section for use with Postgres.

There are other good libraries for database connectivity like Persistent but I like sqlite-simple and it has a gentle learning curve so that is what we will use here. You will learn the basics of database connectivity in this and the next section. Setting up and using sqlite is easy because the sqlite-simple library includes the compiled code for sqlite so configuration requires only the file path to the database file.

 1 {-# LANGUAGE OverloadedStrings #-}\n 2 \n 3 module Main where\n 4   \n 5 import Database.SQLite.Simple\n 6 \n 7 {-\n 8    Create sqlite database:\n 9      sqlite3 test.db \"create table test (id integer primary key, str text);\"\n10 \n11    This example is derived from the example at github.com/nurpax/sqlite-simple\n12 -}\n13 \n14 main :: IO ()\n15 main = do\n16   conn <- open \"test.db\"\n17   -- start by getting table names in database:\n18   do\n19     r <- query_ conn\n20       \"SELECT name FROM sqlite_master WHERE type='table'\" :: IO [Only String]\n21     print \"Table names in database test.db:\"\n22     mapM_ (print . fromOnly) r\n23   \n24   -- get the metadata for table test in test.db:\n25   do\n26     r <- query_ conn\n27           \"SELECT sql FROM sqlite_master WHERE type='table' and name='test'\" ::\n28                 IO [Only String]\n29     print \"SQL to create table 'test' in database test.db:\"\n30     mapM_ (print . fromOnly) r\n31   \n32   -- add a row to table 'test' and then print out the rows in table 'test':\n33   do\n34     execute conn \"INSERT INTO test (str) VALUES (?)\"\n35       (Only (\"test string 2\" :: String))\n36     r2 <- query_ conn \"SELECT * from test\" :: IO [(Int, String)]\n37     print \"number of rows in table 'test':\"\n38     print (length r2)\n39     print \"rows in table 'test':\"\n40     mapM_ print  r2\n41     \n42   close conn\n

\n\n

This Haskell code interacts with an SQLite database named “test.db” using the Database.SQLite.Simple library.

Functionality:

\n
Imports Database.SQLite.Simple: Includes necessary functions for working with SQLite databases.
\n
\n
\n
main Function:
\n
\n
\n
- \nOpens a connection to “test.db”.\n
- \n
  \nRetrieves and prints table names:\n
  \n
  \n
  - \nExecutes an SQL query to get table names from the sqlite_master table.\n
  - \nPrints the table names using mapM_.\n
  \n
  \n
- \n
  \nRetrieves and prints SQL to create ‘test’ table:\n
  \n
  \n
  - \nQueries the sqlite_master table for the SQL used to create the ‘test’ table.\n
  - \nPrints the SQL statement.\n
  \n
  \n
- \n
  \nInserts a row and prints table data:\n
  \n
  \n
  - \nInserts a new row with the string “test string 2” into the ‘test’ table.\n
  - \nSelects all rows from the ‘test’ table.\n
  - \nPrints the number of rows and the rows themselves.\n
  \n
  \n
- \nCloses the database connection.\n
\n
\n

Key Points:

\nDemonstrates basic database interaction using Haskell and SQLite.\n
\nquery_ is used to execute SELECT queries, and execute is used for INSERT queries.\n
\nThe code assumes the existence of the “test.db” database and the “test” table with the specified schema.\n

The type Only used in line 20 acts as a container for a single value and is defined in the simple-sqlite library. It can also be used to pass values for queries like:

r <- query_ conn \"SELECT name FROM customers where id = ?\" (Only 4::Int)\n

\n\n

To run this example start by creating a sqlite database that is stored in the file test.db:

sqlite3 test.db \"create table test (id integer primary key, str text);\"\n

\n\n

Then build and run the example:

stack build --exec TestSqLite1\n

\n\n

Database Access for Postgres

Setting up and using a database in the last section was easy because the sqlite-simple library includes the compiled code for sqlite so configuration only requires the file path the the database file. The Haskel examples for Postgres will be similar to those for Sqlite. There is some complication in setting up Postgres if you do not already have it installed and configured.

In any case, you will need to have Postgres installed and set up with a user account for yourself. When I am installing and configuring Postgres on my Linux laptop, I create a database role markw. You will certainly create a different role/account name so subsitute your role name for markw in the following code examples.

If you are using Ubuntu you can install Postgres and create a role using:

sudo apt-get update\nsudo apt-get install postgresql postgresql-contrib postgresql-server-dev-9.5\nsudo -u postgres createuser --interactive\nEnter name of role to add: markw\nShall the new role be a superuser? (y/n) y\n

\n\n

We will need to install postgresql-server-dev-9.5 in order to use the Haskell Postgres bindings. Note that your version of Ubuntu Linux may have a different version of the server dev package which you can find using:

aptitude search postgresql-dev\n

\n\n

If you are using Mac OS X you can then install Postgres as an application which is convenient for development. A role is automatically created with the same name as your OS X “short name.” You can use the “Open psql” button on the interface to open a command line shell that functions like the psql command on Ubuntu (or other Linux distributions).

aptitude search postgresql-dev\n

\n\n

You will then want to create a database named haskell and set the password for role/account markw to test1 for running the example in this section:

createdb haskell\nsudo -u postgres psql\npostgres=# alter user markw encrypted password 'test1';\npostgres=# \\q\n\npsql -U markw haskell\npsql (9.5.4)\nType \"help\" for help.\n\nhaskell=# create table customers (id int, name text, email text);\nCREATE TABLE\nhaskell=#  insert into customers values (1, 'Acme Cement', 'info@acmecement.com');\nINSERT 0 1\nhaskell=# \\q\n

\n\n

If you are not familiar with using Postgres then take a minute to experiment with using the psql command line utility to connect to the database you just created and peform practice queries:

markw=# \\c haskell\nYou are now connected to database \"haskell\" as user \"markw\".\nhaskell=# \\d\n         List of relations\n Schema |   Name    | Type  | Owner \n--------+-----------+-------+-------\n public | customers | table | markw\n public | links     | table | markw\n public | products  | table | markw\n(3 rows)\n\nhaskell=# select * from customers;\n id |      name       |        email        \n----+-----------------+---------------------\n  1 | Acme Cement     | info@acmecement.com\n  2 | Biff Home Sales | info@biff.com\n  3 | My Pens         | info@mypens.com\n(3 rows)\n\nhaskell=# select * from products;\n id |     name      | cost \n----+---------------+------\n  1 | Cement bag    |  2.5\n  2 | Cheap Pen     |  1.5\n  3 | Expensive Pen | 14.5\n(3 rows)\n\nhaskell=# select * from links;\n id | customer_id | productid \n----+-------------+-----------\n  1 |           1 |         1\n  2 |           3 |         2\n  3 |           3 |         3\n(3 rows)\n\nhaskell=# \n

\n\n

You can change default database settings using ConnectInfo:

ConnectInfo  \n  connectHost :: String\n  connectPort :: Word16\n  connectUser :: String\n  connectPassword :: String\n  connectDatabase :: String\n

\n\n

In the following example on lines 9-10 I use defaultConnectInfo that lets me override just some settings, leaving the rest set at default values. The code to access a database using simple-postgresql is similar to that in the last section, with a few API changes.

 1 {-# LANGUAGE OverloadedStrings #-}\n 2 \n 3 module Main where\n 4   \n 5 import Database.PostgreSQL.Simple\n 6 \n 7 main :: IO ()\n 8 main = do\n 9   conn <- connect defaultConnectInfo { connectDatabase = \"haskell\",\n10                                        connectUser = \"markw\" }\n11   -- start by getting table names in database:\n12   do\n13     r <- query_ conn \"SELECT name FROM customers\" :: IO [(Only String)]\n14     print \"names and emails in table 'customers' in database haskell:\"\n15     mapM_ (print . fromOnly) r\n16   \n17   -- add a row to table 'test' and then print out the rows in table 'test':\n18   do\n19     let rows :: [(Int, String, String)]\n20         rows = [(4, \"Mary Smith\", \"marys@acme.com\")]\n21     executeMany conn\n22       \"INSERT INTO customers (id, name, email) VALUES (?,?,?)\" rows\n23     r2 <- query_ conn \"SELECT * from customers\" :: IO [(Int, String, String)]\n24     print \"number of rows in table 'customers':\"\n25     print (length r2)\n26     print \"rows in table 'customers':\"\n27     mapM_ print  r2\n28     \n29   close conn\n

\n\n

This Haskell code interacts with a PostgreSQL database named “haskell”. It utilizes the Database.PostgreSQL.Simple library to establish a connection, retrieve and insert data into a “customers” table.

Code Breakdown

\n
Import Database.PostgreSQL.Simple: Imports necessary functions for working with PostgreSQL databases.
\n
\n
\n
main Function:
\n
\n
\n
- \n
  \n
  Establishes a connection:
  \n
  \n
  \n
  - \nConnects to the “haskell” database using the default connection information.\n
  - \nThe connection specifies the username as “markw”.\n
  \n
  \n
- \n
  \n
  Retrieves and prints data from the “customers” table:
  \n
  \n
  \n
  - \nExecutes an SQL query to fetch names from the “customers” table.\n
  - \nPrints the retrieved names.\n
  \n
  \n
- \n
  \n
  Inserts a row and prints table data:
  \n
  \n
  \n
  - \nPrepares a list of tuples rows representing the new data to be inserted.\n
  - \nExecutes an SQL INSERT query to add the new row to the “customers” table.\n
  - \nSelects all rows from the “customers” table.\n
  - \nPrints the number of rows and the rows themselves.\n
  \n
  \n
- \n
  \n
  Closes the database connection:
  \n
  \n
  \n
  - \nTerminates the established connection.\n
  \n
  \n
\n
\n

Key Points

\nThe code assumes a “customers” table with columns: id (integer), name (string), and email (string).\n
\nquery_ is used for SELECT queries, and executeMany is used for bulk INSERT queries.\n
\nThe code provides a basic illustration of database interaction in Haskell using the Database.PostgreSQL.Simple library.\n

The type Only used in line 13 acts as a container for a single value that is defined in the simple-postgresql library. It can also be used to pass values for queries like:

r <- query_ conn \"SELECT name FROM customers where id = ?\" (Only 4::Int)\n

\n\n

The monad mapping function mapM_ using in line 15 is like mapM but is used when we do not need the resulting collection from executing the map operation. mapM_ is used for side effects, in this case extracting the value for a collection of Only values and printing them. I removed some output from building the example in the following listing:

$ Database-postgres git:(master) > stack build --exec TestPostgres1\nTestDatabase-0.1.0.0: build\nPreprocessing executable 'TestPostgres1' for TestDatabase-0.1.0.0...\n[1 of 1] Compiling Main             ( TestPostgres1.hs, \n\n\"names and emails in table 'customers' in database haskell:\"\n\"Acme Cement\"\n\"Biff Home Sales\"\n\"My Pens\"\n\"number of rows in table 'customers':\"\n4\n\"rows in table 'customers':\"\n(1,\"Acme Cement\",\"info@acmecement.com\")\n(2,\"Biff Home Sales\",\"info@biff.com\")\n(3,\"My Pens\",\"info@mypens.com\")\n(4,\"Mary Smith\",\"marys@acme.com\")\n

\n\n

Postgres is my default database and I use it unless there is a compelling reason not to. While work for specific customers has mandated using alternative data stores (e.g., BigTable while working at Google and MongoDB at Compass Labs), Postgres supports relational tables, free text search, and structured data like JSON.

Haskell Program to Play the Blackjack Card Game

For much of my work using Haskell I deal mostly with pure code with smaller bits of impure code for network and file IO, etc. Realizing that my use case for using Haskell (mostly pure code) may not be typical, I wanted the last example “cookbook recipe” in this book to be an example dealing with changing state, a program to play the Blackjack card game.

The game state is maintained in the type Table that holds information on a randomized deck of cards, the number of players in addition to the game user and the card dealer, the cards in the current hand, and the number of betting chips that all players own. Table data is immutable so all of the major game playing functions take a table and any other required inputs, and generate a new table as the function result.

This example starts by asking how many players, besides the card dealer and the game user, should play a simulated Blackjack game. The game user controls when they want another card while the dealer and any other simulated players play automatically (they always hit when their card score is less than 17).

I define the types for playing cards and an entire card deck in the file Card.hs:

 1 module Card (Card, Rank, Suit, orderedCardDeck, cardValue) where\n 2 \n 3 import Data.Maybe (fromMaybe)\n 4 import Data.List (elemIndex)\n 5 import Data.Map (fromList, lookup, keys)\n 6 \n 7 data Card = Card { rank :: Rank\n 8                  , suit :: Suit }\n 9                  deriving (Eq, Show)\n10                  \n11 data Suit = Hearts | Diamonds | Clubs | Spades\n12           deriving (Eq, Show, Enum, Ord)\n13 \n14 data Rank = Two | Three | Four\n15           | Five | Six | Seven | Eight\n16           | Nine | Ten | Jack  | Queen | King | Ace\n17           deriving (Eq, Show, Enum, Ord)\n18 \n19 rankMap = fromList [(Two,2), (Three,3), (Four,4), (Five,5),\n20                     (Six,6), (Seven,7), (Eight,8), (Nine,9),\n21                     (Ten,10), (Jack,10), (Queen,10),\n22                     (King,10), (Ace,11)]\n23 \n24 orderedCardDeck :: [Card]\n25 orderedCardDeck = [Card rank suit | rank <- keys rankMap,\n26                                     suit <- [Hearts .. Clubs]]\n27 \n28 cardValue :: Card -> Int\n29 cardValue aCard =\n30   case (Data.Map.lookup (rank aCard) rankMap) of\n31     Just n -> n\n32     Nothing -> 0 -- should never happen\n

\n\n

This module defines essential components for representing and working with playing cards.

Data Types

\n
\n
Card: A record type with two fields:
\n
\n
\n
- \nrank :: Rank - Represents the card’s rank (e.g., Two, Queen, Ace).\n
- \nsuit :: Suit - Represents the card’s suit (e.g., Hearts, Spades).\n
\n
\n
\n
Suit: An enumeration defining the four card suits: Hearts, Diamonds, Clubs, Spades. It derives Eq (equality), Show (string representation), Enum (enumeration capabilities), and Ord (ordering) for convenience.
\n
\n
Rank: An enumeration listing the thirteen card ranks, from Two to Ace. It also derives Eq, Show, Enum, and Ord.
\n

Functions and Values

\n
rankMap: A Data.Map that associates each Rank with its corresponding numerical value in games like Blackjack.
\n
\n
orderedCardDeck: A list comprehension that generates a standard 52-card deck, sorted by rank within each suit.
\n
\n
cardValue: A function that takes a Card and returns its numerical value based on the rankMap. It uses pattern matching to handle the Maybe type returned by Data.Map.lookup.
\n

Explanation

\n
Card Data Type: The core of the module. It defines a playing card as a combination of a Rank and a Suit.
\n
\n
Suit and Rank Enumerations: These provide a clear and type-safe representation of suits and ranks. Deriving Enum and Ord allows easy iteration and comparison.
\n
\n
rankMap: This map is crucial for assigning numerical values to cards, particularly in games where card values matter (e.g., Blackjack).
\n
\n
orderedCardDeck: This function generates a standard 52-card deck. It uses list comprehension to iterate over all Rank values (obtained from the keys of rankMap) and all Suit values (from Hearts to Clubs), creating a Card for each combination.
\n
\n
cardValue: This function retrieves the numerical value of a given card. It uses Data.Map.lookup to find the value associated with the card’s rank in rankMap. The case expression handles the possibility of lookup returning Nothing (which should ideally never happen in this context).
\n

Key Points

\nThe code provides a well-structured representation of playing cards in Haskell.\n
\nThe use of enumerations enhances type safety and readability.\n
\nData.Map is employed for efficient lookup of card values.\n
\nThe orderedCardDeck function conveniently generates a standard deck of cards.\n

As usual, the best way to understand this code is to go to the GHCi repl:

 1 *Main Card RandomizedList Table> :l Card\n 2 [1 of 1] Compiling Card             ( Card.hs, interpreted )\n 3 Ok, modules loaded: Card.\n 4 *Card> :t orderedCardDeck\n 5 orderedCardDeck :: [Card]\n 6 *Card> orderedCardDeck\n 7 [Card {rank = Two, suit = Hearts},Card {rank = Two, suit = Diamonds},Card {rank = Tw\\\n 8 o, suit = Clubs},Card {rank = Three, suit = Hearts},Card {rank = Three,\n 9     ...\n10 *Card> head orderedCardDeck\n11 Card {rank = Two, suit = Hearts}\n12 *Card> cardValue $ head orderedCardDeck\n13 2\n

\n\n

So, we have a sorted deck of cards and a utility function for returning the numerical value of a card (we always count ace cards as 11 points, deviating from standard Blackjack rules).

The next thing we need to get is randomly shuffled lists. The Haskell Wiki has a good writeup on randomizing list elements and we are borrowing their function randomizedList (you can see the source code in the file RandomizedList.hs). Here is a sample use:

 1 *Card> :l RandomizedList.hs \n 2 [1 of 1] Compiling RandomizedList   ( RandomizedList.hs, interpreted )\n 3 Ok, modules loaded: RandomizedList.\n 4 *RandomizedList> import Card\n 5 *RandomizedList Card> randomizedList orderedCardDeck\n 6 [Card {rank = Queen, suit = Hearts},Card {rank = Six, suit = Diamonds},Card {rank = \\\n 7 Five, suit = Clubs},Card {rank = Five, suit = Diamonds},Card {rank = Seven, suit = C\n 8 lubs},Card {rank = Three, suit = Hearts},Card {rank = Four, suit = Diamonds},Card {r\n 9 ank = Ace, suit = Hearts},\n10   ...\n

\n\n

Much of the complexity in this example is implemented in Table.hs which defines the type Table and several functions to deal and score hands of dealt cards:

\ncreateNewTable :: Players -> Table. Players is the integer number of other players at the table.\n
\nsetPlayerBet :: Int -> Table -> Table. Given a new value to bet and a table, generate a new modified table.\n
\nshowTable :: Table -> [Char]. Given a table, generate a string describing the table (in a format useful for development)\n
\ninitialDeal :: [Card] -> Table -> Int -> Table. Given a randomized deck of cards, a table, and the number of other players, generate a new table.\n
\nchangeChipStack :: Int -> Int -> Table -> Table. Given a player index (index order: user, dealer, and other players), a new number of betting chips for the player, and a table, then generate a new modified table.\n
\nsetCardDeck :: [Card] -> Table -> Table. Given a randomized card deck and a table, generate a new table containing the new randomized card list; all other table data is unchanged.\n
\ndealCards :: Table -> [Int] -> Table. Given a table and a list of player indices for players wanting another card, generate a new modified table.\n
\nresetTable :: [Card] -> Table -> Int -> Table. Given a new randomized card deck, a table, and a new number of other players, generate a new table.\n
\nscoreHands :: Table -> Table. Given a table, score all dealt hands and generate a new table with these scores. There is no table type score data, rather, we “score” by changing the number of chips all of the players (inclding the dealer) has.\n
\ndealCardToUser :: Table -> Int -> Table. For the game user, always deal a card. For the dealer and other players, deal another card if their hand score is less than 17.\n
\nhandOver :: Table -> Bool. Determine if the current hand is over.\n
\nsetPlayerPasses :: Table -> Table. Call this function when the payer passes. Other players and dealer are then played out automatically.\n

The implementation in the file Table.hs is fairly simple, with the exception of the use of Haskell lenses to access nested data in the table type. I will discuss the use of lenses after the program listing, but: as you are reading the code look out for variables starting with the underscore character _ that alerts the Lens system that it should create data accessors for these variables.

This code defines a module named Table which provides data structures and functions to simulate a simplified table in a card game, potentially Blackjack.

Core Components

\n
\n
Table data type:
\n
\n
\n
- \n
  \nRepresents the state of the table, storing information like:\n
  \n
  \n
  - \nNumber of players\n
  - \nChip stacks for each player\n
  - \nCards dealt to each player (including the dealer)\n
  - \nCurrent player’s bet\n
  - \nWhether the user has passed their turn\n
  - \nThe remaining card deck\n
  \n
  \n
\n
\n
\n
\n
Functions:
\n
\n
\n
- \ncreateNewTable: Creates a new table with the specified number of players and initial chip stacks.\n
- \nresetTable: Resets the table for a new round, clearing dealt cards and optionally changing the card deck.\n
- \nsetCardDeck: Sets a new card deck for the table.\n
- \ndealCards: Deals cards to specified players.\n
- \ninitialDeal: Performs the initial deal at the beginning of a round.\n
- \nshowTable: Generates a string representation of the table’s current state.\n
- \nscoreHands: Calculates and updates chip stacks based on player and dealer scores.\n
- \nsetPlayerBet: Sets the current player’s bet.\n
- \nsetPlayerPasses: Simulates the player passing their turn, dealing additional cards to other players and the dealer.\n
- \nchangeChipStack: Modifies a specific player’s chip stack.\n
- \nscore: Calculates the score of a player’s hand.\n
- \ndealCardToUser: Deals a card to a specified player, with special handling for the user and dealer.\n
- \nhandOver: Checks if the user has passed their turn.\n
\n
\n

Lenses

The code uses lenses (makeLenses ''Table) to provide convenient access and modification of the Table data type’s fields.

Game Logic (Simplified)

\nThe code seems to implement a basic version of a card game where players and the dealer are dealt cards.\n
\nscoreHands calculates scores and updates chip stacks based on win/loss conditions.\n
\ndealCardToUser handles dealing cards, ensuring the dealer keeps drawing until their score is at least 17.\n
\nsetPlayerPasses simulates the user passing, triggering the dealer and other players to finish their turns.\n

  1 {-# LANGUAGE TemplateHaskell #-}  -- for makeLens\n  2 \n  3 module Table (Table (..), createNewTable, setPlayerBet, showTable, initialDeal,\n  4               changeChipStack, setCardDeck, dealCards, resetTable, scoreHands,\n  5               dealCardToUser, handOver, setPlayerPasses) where\n  6   -- note: export dealCardToUser only required for ghci development\n  7 \n  8 import Control.Lens\n  9 \n 10 import Card\n 11 import Data.Bool\n 12 import Data.Maybe (fromMaybe)\n 13 \n 14 data Table = Table { _numPlayers :: Int\n 15                    , _chipStacks :: [Int] -- number of chips,\n 16                                           -- indexed by player index\n 17                    , _dealtCards :: [[Card]] -- dealt cards for user,\n 18                                              -- dealer, and other players\n 19                    , _currentPlayerBet :: Int\n 20                    , _userPasses       :: Bool\n 21                    , _cardDeck         :: [Card]\n 22                    }\n 23            deriving (Show)\n 24            \n 25 type Players = Int\n 26              \n 27 createNewTable :: Players -> Table\n 28 createNewTable n =\n 29   Table n\n 30         [500 | _ <- [1 .. n]] -- give each player (incuding dealer) 10 chips\n 31         [[] | _ <- [0..n]] -- dealt cards for user and other players\n 32                            -- (we don't track dealer's chips)\n 33         20 -- currentPlayerBet number of betting chips\n 34         False\n 35         [] -- placeholder for random shuffled card deck\n 36  \n 37 resetTable :: [Card] -> Table -> Int -> Table\n 38 resetTable cardDeck aTable numberOfPlayers =\n 39   Table numberOfPlayers\n 40         (_chipStacks aTable)       -- using Lens accessor\n 41         [[] | _ <- [0..numberOfPlayers]]\n 42         (_currentPlayerBet aTable) -- using Lens accessor\n 43         False\n 44         cardDeck\n 45      \n 46      -- Use lens extensions for type Table:\n 47             \n 48 makeLenses ''Table\n 49  \n 50 showDealtCards :: [[Card]] -> String\n 51 showDealtCards dc =\n 52   (show [map cardValue hand | hand <- dc])\n 53 \n 54 setCardDeck :: [Card] -> Table -> Table\n 55 setCardDeck newDeck =\n 56   over cardDeck (\\_ -> newDeck)  -- change value to new card deck\n 57 \n 58 dealCards :: Table -> [Int] -> Table\n 59 dealCards aTable playerIndices =\n 60   last $ scanl dealCardToUser aTable playerIndices\n 61  \n 62 initialDeal cardDeck aTable numberOfPlayers =\n 63   dealCards\n 64     (dealCards (resetTable cardDeck aTable numberOfPlayers)\n 65                [0 .. numberOfPlayers])\n 66     [0 .. numberOfPlayers]\n 67     \n 68 showTable :: Table -> [Char]\n 69 showTable aTable =\n 70   \"\\nCurrent table data:\\n\" ++\n 71   \"  Chipstacks: \" ++\n 72   \"\\n    Player: \" ++ (show (head (_chipStacks aTable))) ++\n 73   \"\\n    Other players: \" ++ (show (tail (_chipStacks aTable))) ++\n 74   \"\\n  User cards: \" ++ (show (head (_dealtCards aTable))) ++\n 75   \"\\n  Dealer cards: \" ++ (show ((_dealtCards aTable) !! 1)) ++\n 76   \"\\n  Other player's cards: \" ++ (show (tail (tail(_dealtCards aTable)))) ++\n 77   -- \"\\n  Dealt cards: \" ++ (show (_dealtCards aTable)) ++\n 78   \"\\n  Dealt card values: \" ++ (showDealtCards (_dealtCards aTable)) ++\n 79   \"\\n  Current player bet: \" ++\n 80   (show (_currentPlayerBet aTable)) ++\n 81   \"\\n  Player pass: \" ++\n 82   (show (_userPasses aTable)) ++ \"\\n\"\n 83   \n 84 clipScore aTable playerIndex =\n 85   let s = score aTable playerIndex in\n 86     if s < 22 then s else 0\n 87       \n 88 scoreHands aTable =\n 89   let chipStacks2 = _chipStacks aTable\n 90       playerScore = clipScore aTable 0\n 91       dealerScore = clipScore aTable 1\n 92       otherScores = map (clipScore aTable) [2..]\n 93       newPlayerChipStack = if playerScore > dealerScore then\n 94                              (head chipStacks2) + (_currentPlayerBet aTable)\n 95                            else\n 96                              if playerScore < dealerScore then\n 97                                 (head chipStacks2) - (_currentPlayerBet aTable)\n 98                              else (head chipStacks2)\n 99       newOtherChipsStacks =\n100         map (\\(x,y) -> if x > dealerScore then\n101                          y + 20\n102                        else\n103                          if x < dealerScore then\n104                            y - 20\n105                          else y) \n106             (zip otherScores (tail chipStacks2))\n107       newChipStacks  = newPlayerChipStack:newOtherChipsStacks\n108   in\n109     over chipStacks (\\_ -> newChipStacks) aTable\n110      \n111 setPlayerBet :: Int -> Table -> Table\n112 setPlayerBet newBet =\n113   over currentPlayerBet (\\_ -> newBet)  \n114 \n115 setPlayerPasses :: Table -> Table\n116 setPlayerPasses aTable =\n117   let numPlayers = _numPlayers aTable\n118       playerIndices = [1..numPlayers]\n119       t1 = over userPasses (\\_ -> True) aTable\n120       t2 = dealCards t1 playerIndices\n121       t3 = dealCards t2 playerIndices\n122       t4 = dealCards t3 playerIndices\n123   in\n124     t4\n125     \n126     \n127 changeChipStack :: Int -> Int -> Table -> Table\n128 changeChipStack playerIndex newValue =\n129   over chipStacks (\\a -> a & element playerIndex .~ newValue)\n130 \n131 scoreOLD aTable playerIndex =\n132   let scores = map cardValue ((_dealtCards aTable) !! playerIndex)\n133       totalScore = sum scores in\n134     if totalScore < 22 then totalScore else 0\n135 \n136 score aTable playerIndex =\n137   let scores = map cardValue ((_dealtCards aTable) !! playerIndex)\n138       totalScore = sum scores in\n139     totalScore\n140   \n141 dealCardToUser' :: Table -> Int -> Table\n142 dealCardToUser' aTable playerIndex =\n143   let nextCard = head $ _cardDeck aTable\n144       playerCards = nextCard : ((_dealtCards aTable) !! playerIndex)\n145       newTable = over cardDeck (\\cd -> tail cd) aTable in\n146     over dealtCards (\\a -> a & element playerIndex .~ playerCards) newTable\n147 \n148 dealCardToUser :: Table -> Int -> Table\n149 dealCardToUser aTable playerIndex\n150   | playerIndex == 0  = dealCardToUser' aTable playerIndex -- user\n151   | otherwise         = if (score aTable playerIndex) < 17 then\n152                              dealCardToUser' aTable playerIndex\n153                         else aTable\n154   \n155 handOver :: Table -> Bool\n156 handOver aTable =\n157   _userPasses aTable\n

\n\n

In line 48 we use the function makeLenses to generate access functions for the type Table. We will look in some detail at lines 54-56 where we use the lense over function to modify a nested value in a table, returning a new table:

1 setCardDeck :: [Card] -> Table -> Table\n2 setCardDeck newDeck =\n3   over cardDeck (\\_ -> newDeck)\n

\n\n

The expression in line 3 evaluates to a partial function that takes another argument, a table, and returns a new table with the card deck modified. Function over expects a function as its second argument. In this example, the inline function ignores the argument it is called with, which would be the old card deck value, and returns the new card deck value which is placed in the table value.

Using lenses can greatly simplify the code to manipulate complex types.

Another place where I am using lenses is in the definition of function scoreHands (lines 88-109). On line 109 we are using the over function to replace the old player betting chip counts with the new value we have just calculated:

1   over chipStacks (\\_ -> newChipStacks) aTable\n

\n\n

Similarly, we use over in line 113 to change the current player bet. In function handOver on line 157, notice how I am using the generated function _userPasses to extract the value of the user passes boolean flag from a table.

The function main, defined in the file Main.hs, uses the code we have just seen to represent a table and modify a table, is fairly simple. A main game loop repetitively accepts game user input, and calls the appropriate functions to modify the current table, producing a new table. Remember that the table data is immutable: we always generate a new table from the old table when we need to modify it.

 1 module Main where\n 2 \n 3 import Card   -- pure code\n 4 import Table  -- pure code\n 5 import RandomizedList  -- impure code\n 6 \n 7 printTable :: Table -> IO ()\n 8 printTable aTable =\n 9   putStrLn $ showTable aTable\n10   \n11 randomDeck =\n12   randomizedList orderedCardDeck\n13 \n14 gameLoop :: Table -> Int -> IO b\n15 gameLoop aTable numberOfPlayers = do\n16   printTable aTable\n17   cardDeck <- randomDeck\n18   if (handOver aTable) then\n19     do\n20       putStrLn \"\\nHand over. State of table at the end of the game:\\n\"\n21       printTable aTable\n22       putStrLn \"\\nNewly dealt hand:\\n\"\n23       gameLoop (initialDeal cardDeck (scoreHands aTable)\n24                                      numberOfPlayers)\n25                                      numberOfPlayers\n26   else\n27     do\n28       putStrLn \"Enter command:\"\n29       putStrLn \"  h)it or set bet to 10, 20, 30; any other key to stay:\"\n30       command <- getLine\n31       if elem command [\"10\", \"20\", \"30\"] then\n32         gameLoop (setPlayerBet (read command) aTable) numberOfPlayers\n33       else\n34         if command == \"h\" then\n35           gameLoop (dealCards aTable [0 .. numberOfPlayers]) numberOfPlayers\n36         else\n37           gameLoop (setPlayerPasses (dealCards aTable [1 .. numberOfPlayers]))\n38                     numberOfPlayers \n39              -- player stays (no new cards)\n40   \n41 main :: IO b\n42 main = do\n43   putStrLn \"Start a game of Blackjack. Besides yourself, how many other\"\n44   putStrLn \"players do you want at the table?\"\n45   s <- getLine\n46   let num = (read s :: Int) + 1\n47   cardDeck <- randomDeck\n48   let aTable = initialDeal cardDeck (createNewTable num) num\n49   gameLoop aTable num\n

\n\n

This module combines the previously defined Card and Table modules with an impure RandomizedList module to implement the main game loop of a simplified Blackjack-like card game.

Core Functions

\n
printTable: Prints the current state of the table using the showTable function from the Table module.
\n
\n
randomDeck: Generates a randomized version of the orderedCardDeck using the randomizedList function (assumed to be from the RandomizedList module).
\n
\n
\n
gameLoop: The core recursive function that drives the game:
\n
\n
\n
- \nPrints the current table state.\n
- \nGenerates a random card deck.\n
- \nIf the hand is over (user has passed), prints the final table state, scores hands, and starts a new game with the updated chip stacks.\n
- \n
  \nOtherwise, prompts the user for a command:\n
  \n
  \n
  - \nIf the command is “10”, “20”, or “30”, sets the player’s bet.\n
  - \nIf the command is “h”, deals cards to all players (including the user).\n
  - \nIf any other command is entered, sets the user as passed and deals cards to the dealer and other players until they stand.\n
  \n
  \n
- \nRecursively calls itself with the updated table state.\n
\n
\n
\n
\n
main:
\n
\n
\n
- \nPrompts the user for the number of additional players.\n
- \nCreates a new table with the specified number of players and an initial deal.\n
- \nStarts the gameLoop.\n
\n
\n

Key Points

\nThe code demonstrates a basic interactive text-based card game implementation.\n
\nIt combines pure modules (Card, Table) with an impure module (RandomizedList) for randomization.\n
\nThe gameLoop function handles user input and game state transitions.\n
\nThe game logic is likely simplified for this example, and a full Blackjack implementation would require additional rules and features.\n

Remember: The RandomizedList module is assumed to provide a function randomizedList for shuffling the card deck, introducing impurity into the game logic.

I encourage you to try playing the game yourself, but if you don’t here is a sample game:

 1 *Main Card RandomizedList Table> main\n 2 Start a game of Blackjack. Besides yourself, how many other\n 3 players do you want at the table?\n 4 1\n 5 \n 6 Current table data:\n 7   Chipstacks: \n 8     Player: 500\n 9     Other players: [500]\n10   User cards: [Card {rank = Three, suit = Clubs},Card {rank = Two, suit = Hearts}]\n11   Dealer cards: [Card {rank = Queen, suit = Diamonds},Card {rank = Seven, suit = Clu\\\n12 bs}]\n13   Other player's cards: [[Card {rank = King, suit = Hearts},Card {rank = Six, suit =\\\n14  Diamonds}]]\n15   Dealt card values: [[3,2],[10,7],[10,6]]\n16   Current player bet: 20\n17   Player pass: False\n18 \n19 Enter command: h)it or set bet to 10, 20, 30; any other key to stay:\n20 h\n21 \n22 Current table data:\n23   Chipstacks: \n24     Player: 500\n25     Other players: [500]\n26   User cards: [Card {rank = Six, suit = Hearts},Card {rank = Three, suit = Clubs},Ca\\\n27 rd {rank = Two, suit = Hearts}]\n28   Dealer cards: [Card {rank = Queen, suit = Diamonds},Card {rank = Seven, suit = Clu\\\n29 bs}]\n30   Other player's cards: [[Card {rank = Eight, suit = Hearts},Card {rank = King, suit\\\n31  = Hearts},Card {rank = Six, suit = Diamonds}]]\n32   Dealt card values: [[6,3,2],[10,7],[8,10,6]]\n33   Current player bet: 20\n34   Player pass: False\n35 \n36 Enter command: h)it or set bet to 10, 20, 30; any other key to stay:\n37 h\n38 \n39 Current table data:\n40   Chipstacks: \n41     Player: 500\n42     Other players: [500]\n43   User cards: [Card {rank = King, suit = Clubs},Card {rank = Six, suit = Hearts},Car\\\n44 d {rank = Three, suit = Clubs},Card {rank = Two, suit = Hearts}]\n45   Dealer cards: [Card {rank = Queen, suit = Diamonds},Card {rank = Seven, suit = Clu\\\n46 bs}]\n47   Other player's cards: [[Card {rank = Eight, suit = Hearts},Card {rank = King, suit\\\n48  = Hearts},Card {rank = Six, suit = Diamonds}]]\n49   Dealt card values: [[10,6,3,2],[10,7],[8,10,6]]\n50   Current player bet: 20\n51   Player pass: False\n52 \n53 Enter command: h)it or set bet to 10, 20, 30; any other key to stay:\n54 \n55 Current table data:\n56   Chipstacks: \n57     Player: 500\n58     Other players: [500]\n59   User cards: [Card {rank = King, suit = Clubs},Card {rank = Six, suit = Hearts},Car\\\n60 d {rank = Three, suit = Clubs},Card {rank = Two, suit = Hearts}]\n61   Dealer cards: [Card {rank = Queen, suit = Diamonds},Card {rank = Seven, suit = Clu\\\n62 bs}]\n63   Other player's cards: [[Card {rank = Eight, suit = Hearts},Card {rank = King, suit\\\n64  = Hearts},Card {rank = Six, suit = Diamonds}]]\n65   Dealt card values: [[10,6,3,2],[10,7],[8,10,6]]\n66   Current player bet: 20\n67   Player pass: True\n68 \n69 Hand over. State of table at the end of the game:\n70 \n71 Current table data:\n72   Chipstacks: \n73     Player: 520\n74     Other players: [520]\n75   User cards: [Card {rank = King, suit = Clubs},Card {rank = Six, suit = Hearts},Car\\\n76 d {rank = Three, suit = Clubs},Card {rank = Two, suit = Hearts}]\n77   Dealer cards: [Card {rank = Queen, suit = Diamonds},Card {rank = Seven, suit = Clu\\\n78 bs}]\n79   Other player's cards: [[Card {rank = Eight, suit = Hearts},Card {rank = King, suit\\\n80  = Hearts},Card {rank = Six, suit = Diamonds}]]\n81   Dealt card values: [[10,6,3,2],[10,7],[8,10,6]]\n82   Current player bet: 20\n83   Player pass: True\n

\n\n

Here the game user has four cards with values of [10,6,3,2] for a winning score of 21. The dealer has [10,7] for a score of 17 and the other player has [8,10,6], a value greater than 21 so the player went “bust.”

I hope that you enjoyed this last example that demonstrates a reasonable approach for managing state when using immutable data.

Section 3 - Larger Projects

So far we have covered the basics of Haskell programming and seen many examples. In this section we look at a few new projects that I derived from my own work and these new examples will hopefully further encourage you to think of novel uses for Haskell in your own work.

The project knowledge_graph_creator helps to automate the process of creating Knowledge Graphs from raw text input and generates data for both the Neo4J open source graph database as well as RDF data for use in semantic web and linked data applications. I have also implemented this same application in Common Lisp that is also a new example in the latest edition of my book Loving Common Lisp, Or The Savvy Programmer’s Secret Weapon.

The next two chapters in this section are similar in that they both use examples of using Python for Natural Language Processing (NLP) tasks, wrapping the Python code as a REST service, and then writing Haskell clients for these services.

The project HybridHaskellPythonNlp uses web services written in Python for natural language processing. The Python web services use the SpaCy library.

The project HybridHaskellPythonCorefAnaphoraResolution uses web services written in Python to allow Haskell applications to use deep learning models created with TensorFlow and Keras.

In these last two examples I use REST APIs to access code written in Python. A good alternative that I don’t cover in this book is using the servant library for generating distributed applications.

Knowledge Graph Creator

The large project described here processes raw text inputs and generates data for knowledge graphs in formats for both the Neo4J graph database and in RDF format for semantic web and linked data applications.

This application works by identifying entities in text. Example entity types are people, companies, country names, city names, broadcast network names, political party names, and university names. We saw earlier code for detecting entities in the chapter on natural language processing (NLP) and we will reuse this code. We will discuss later three strategies for reusing code from different projects.

The following figure shows part of a Neo4J Knowledge Graph created with the example code. This graph has shortened labels in displayed nodes but Neo4J offers a web browser-based console that lets you interactively explore Knowledge Graphs. We don’t cover setting up Neo4J here so please use the Neo4J documentation. As an introduction to RDF data, the semantic web, and linked data you can get free copies of my two books Practical Semantic Web and Linked Data Applications, Common Lisp Edition and Practical Semantic Web and Linked Data Applications, Java, Scala, Clojure, and JRuby Edition.

\n\n

There are two versions of this project that deal with generating duplicate data in two ways:

\nAs either Neo4J Cypher data or RDF triples data are created, store generated data in a SQLite embedded database. Check this database before writing new output data.\n
\nIgnore the problem of generating duplicate data and filter out duplicates in the outer processing pipeline that uses the Knowledge Graph Creator as one processing step.\n

For my own work I choose the second method since filtering duplicates is as easy as a few Makefile targets (the following listing is in the file Makefile in the directory\nhaskell_tutorial_cookbook_examples/knowledge_graph_creator_pure):

all: gendata rdf cypher\n\ngendata:\n    stack build --fast --exec Dev-exe\n\nrdf:\n    echo \"Removing duplicate RDF statements\"\n    awk '!visited[$$0]++' out.n3 > output.n3\n    rm -f out.n3\n\ncypher:\n    echo \"Removing duplicate Cypher statements\"\n    awk '!visited[$$0]++' out.cypher > output.cypher\n    rm -f out.cypher\n

\n\n

The Haskell KGCreator application we develop here writes output files out.n3 (N3 is a RDF data format) and out.cypher (Cypher is the import output format and query language for the Neo4J open source and commercial graph database). The awk commands remove duplicate lines and write de-duplicated data to output.n3 and output.cypher.

We will use this second approach but the next section provides sufficient information and a link to alternative code in case you are interested in using SQLite to prevent duplicate data generation.

Notes for Using SQLite to Avoid Duplicates (Optional Material)

We saw two methods of avoiding duplicates in generated data in the last section. If you want to use the first method for avoiding generating duplicate data, I leave it as an exercise but here are some notes to get you started: you can then modify the example code by using the utility function Blackboard.h in the directory knowledge_graph_creator_pure/src/fileutils and implement the logic seen below for checking new generated data to see if it is in the SQLite database. This first method as it also is a good example for wrapping the embedded SQLite library in an IO Monad and is left as an exercise, otherwise skip this section.

Before you write either an RDF statement or a Neo4J Cypher data import statement, check to see if the statement has already been written using something like:

  check <- blackboard_check_key new_data_uri\n  if check\n     ....\n

\n\n

and after writing a RDF statement or a Neo4J Cypher data import statement, write it to the temporary SQLite database using something like:

  blackboard_write newStatementString\n

\n\n

For the rest of the chapter we will use the approach of not keeping track of generated data in SQLite and instead remove duplicates during post-processing using the standard awk command line utility.

This section is optional. In the rest of this chapter we use the example code in knowledge_graph_creator_pure.

\n\n

We will reuse the code for finding entities that we studied in an earlier chapter. There are several ways to reuse code from multiple local Haskell projects:

\nIn a project’s cabal file, use relative paths to the source code for other projects. This is my preferred way to work but has the drawback that the stack command sdist to make a distribution tarball will not work with relative paths. If this is a problem for you then create relative symbolic file links to the source directories in other projects.\n
\nIn your project’s stack.yaml file, add the other project’s name and path as a extra-deps.\n
\nIn library projects, define a packages definition and install the library globally on your system.\n

I almost always use the first method on my projects with dependencies on other local projects I work on and this is also the approach we use here. The relevant lines in the file KGCreator.cabal are:

 1 library\n 2   exposed-modules:\n 3       CorefWebClient\n 4       NlpWebClient\n 5       ClassificationWebClient\n 6       DirUtils\n 7       FileUtils\n 8       BlackBoard\n 9       GenTriples\n10       GenNeo4jCypher\n11       Apis\n12       Categorize\n13       NlpUtils\n14       Summarize\n15       Entities\n16   other-modules:\n17       Paths_KGCreator\n18       BroadcastNetworkNamesDbPedia\n19       Category1Gram\n20       Category2Gram\n21       CityNamesDbpedia\n22       CompanyNamesDbpedia\n23       CountryNamesDbpedia\n24       PeopleDbPedia\n25       PoliticalPartyNamesDbPedia\n26       Sentence\n27       Stemmer\n28       TradeUnionNamesDbPedia\n29       UniversityNamesDbPedia\n30 \n31   hs-source-dirs:\n32       src\n33       src/webclients\n34       src/fileutils\n35       src/sw\n36       src/toplevel\n37       ../NlpTool/src/nlp\n38       ../NlpTool/src/nlp/data\n

\n\n

This is a standard looking cabal file except for lines 37 and 38 where the source paths reference the example code for the NlpTool application developed in a previous chapter. The exposed module BlackBoard (line 8) is not used but I leave it in the cabal file in case you want to experiment with recording generated data in SQLite to avoid data duplication. You are likely to also want to use BlackBoard if you modify this example to continuously process incoming data in a production system. This is left as an exercise.

Before going into too much detail on the implementation let’s look at the layout of the project code:

\n\n

 1 src/fileutils:\n 2 BlackBoard.hs   DirUtils.hs FileUtils.hs\n 3 \n 4 ../NlpTool/src/nlp:\n 5 Categorize.hs   Entities.hs NlpUtils.hs Sentence.hs Stemmer.hs  Summarize.hs    data\n 6 \n 7 ../NlpTool/src/nlp/data:\n 8 BroadcastNetworkNamesDbPedia.hs CompanyNamesDbpedia.hs      TradeUnionNamesDbPedia.hs\n 9 Category1Gram.hs        CountryNamesDbpedia.hs      UniversityNamesDbPedia.hs\n10 Category2Gram.hs        PeopleDbPedia.hs\n11 CityNamesDbpedia.hs     PoliticalPartyNamesDbPedia.hs\n12 \n13 src/sw:\n14 GenNeo4jCypher.hs   GenTriples.hs\n15 \n16 src/toplevel:\n17 Apis.hs\n

\n\n

As mentioned before, we are using the Haskell source fies in a relative path ../NlpTool/src/… and the local src directory. We discuss this code in the next few sections.

The Main Event: Detecting Entities in Text

A primary task in KGCreator is to identify entities (people, places, etc.) in text and then we will create RDF and Neo4J Cypher data statements using these entities, knowledge of the origin of text data and general relationships between entities.

We will use the top level code that we developed earlier that is located in the directory ../NlpTool/src/nlp (please see the chapter Natural Language Processing Tools for more detail):

\nCategorize.hs - categorizes text into categories like news, religion, business, politics, science, etc.\n
\nEntities.hs - identifies entities like people, companies, places, new broadcast networks, labor unions, etc. in text\n
\nSummarize.hs - creates an extractive summary of text\n

The KGCreator Haskell application looks in a specified directory for text files to process. For each file with a .txt extension there should be a matching file with the extension .meta that contains a single line: the URI of the web location where the corresponding text was found. The reason we need this is that we want to create graph knowledge data from information found in text sources and the original location of the data is important to preserve. In other words, we want to know where the data elements in our knowledge graph came from.

We have not looked at an example of using command line arguments yet so let’s go into some detail on how we do this.\nPreviously when we have defined an output target executable in our .cabal file,\nin this case KGCreator-exe, we could use stack to build the executable and run it with:

\n\n

1 stack build --fast --exec KGCreator-exe\"\n

\n\n

Now, we have an executable that requires two arguments: a source input directory and the file root for generated RDF and Cypher output files. We can pass command line arguments using this notation:

\n\n

1 stack build --fast --exec \"KGCreator-exe test_data outtest\"\n

\n\n

The two command line arguments are:

\ntest_data which is the file path of a local directory containing the input files\n
\nouttest which is the root file name for generated Neo4J Cypher and RDF output files\n

If you are using KGCreator in production, then you will want to copy the compiled and linked executable file KGCreator-exe to somewhere on your PATH like /usr/local/bin.

The following listing shows the file app/Main.hs, the main program for this example that handles command line arguments and calls two top level functions in src/toplevel/Apis.hs:

 1 module Main where\n 2 \n 3 import System.Environment (getArgs)\n 4 import Apis (processFilesToRdf, processFilesToNeo4j)\n 5 \n 6 main :: IO ()\n 7 main = do\n 8   args <- getArgs\n 9   case args of\n10     [] -> error \"must supply an input directory containing text and meta files\"\n11     [_] -> error \"in addition to an input directory, also specify a root file name f\\\n12 or the generated RDF and Cypher files\"\n13     [inputDir, outputFileRoot] -> do\n14         processFilesToRdf   inputDir $ outputFileRoot ++ \".n3\"\n15         processFilesToNeo4j inputDir $ outputFileRoot ++ \".cypher\"\n16     _ -> error \"too many arguments\"\n

\n\n

Here we use getArgs in line8 to fetch a list of command line arguments and verify that at least two arguments have been provided. Then we call the functions processFilesToRdf and processFilesToNeo4j and the functions they call in the next three sections.

Utility Code for Generating RDF

The code for generating RDF and for generating Neo4J Cypher data is similar. We start with the code to generate RDF triples. Before we look at the code, let’s start with a few lines of generated RDF:

<http://dbpedia.org/resource/The_Wall_Street_Journal> \n  <http://knowledgebooks.com/schema/aboutCompanyName> \n  \"Wall Street Journal\" .\n<https://newsshop.com/june/z902.html>\n  <http://knowledgebooks.com/schema/containsCountryDbPediaLink>\n  <http://dbpedia.org/resource/Canada> .\n

\n\n

The next listing shows the file src/sw/GenTriples.hs that finds entities like broadcast network names, city names, company names, people’s names, political party names, and university names in text and generates RDF triple data. If you need to add more entity types for your own applications, then use the following steps:

\nLook at the format of entity data for the NlpTool example and add names for the new entity type you are adding.\n
\nAdd a utility function to find instances of the new entity type to NlpTools. For example, if you are adding a new entity type “park names”, then copy the code for companyNames to parkNames, modify as necessary, and export parkNames.\n
\nIn the following code, add new code for the new entity helper function after lines 10, 97, 151, and 261. Use the code for companyNames as an example.\n

The map category_to_uri_map* created in lines 36 to 84 maps a topic name to a linked Data URI that describes the topic. For example, we would not refer to an information source as being about the topic “economics”, but would instead refer to a linked data URI like http://knowledgebooks.com/schema/topic/economics. The utility function uri_from_categor takes a text description of a topic like “economy” and converts it to an appropriate URI using the map category_to_uri_map*.

The utility function textToTriple takes a file path to a text input file and a path to meta file path, calculates the text string representing the generated triples for the input text file, and returns the result wrapped in an IO monad.

  1 module GenTriples\n  2   ( textToTriples\n  3   , category_to_uri_map\n  4   ) where\n  5 \n  6 import Categorize (bestCategories)\n  7 import Entities\n  8   ( broadcastNetworkNames\n  9   , cityNames\n 10   , companyNames\n 11   , countryNames\n 12   , peopleNames\n 13   , politicalPartyNames\n 14   , tradeUnionNames\n 15   , universityNames\n 16   )\n 17 import FileUtils\n 18   ( MyMeta\n 19   , filePathToString\n 20   , filePathToWordTokens\n 21   , readMetaFile\n 22   , uri\n 23   )\n 24 import Summarize (summarize, summarizeS)\n 25 \n 26 import qualified Data.Map as M\n 27 import Data.Maybe (fromMaybe)\n 28 \n 29 generate_triple :: [Char] -> [Char] -> [Char] -> [Char]\n 30 generate_triple s p o = s ++ \"  \" ++ p ++ \"  \" ++ o ++ \" .\\n\"\n 31 \n 32 make_literal :: [Char] -> [Char]\n 33 make_literal s = \"\\\"\" ++ s ++ \"\\\"\"\n 34 \n 35 category_to_uri_map :: M.Map [Char] [Char]\n 36 category_to_uri_map =\n 37   M.fromList\n 38     [ (\"news_weather\", \"<http://knowledgebooks.com/schema/topic/weather>\")\n 39     , (\"news_war\", \"<http://knowledgebooks.com/schema/topic/war>\")\n 40     , (\"economics\", \"<http://knowledgebooks.com/schema/topic/economics>\")\n 41     , (\"news_economy\", \"<http://knowledgebooks.com/schema/topic/economics>\")\n 42     , (\"news_politics\", \"<http://knowledgebooks.com/schema/topic/politics>\")\n 43     , (\"religion\", \"<http://knowledgebooks.com/schema/topic/religion>\")\n 44     , ( \"religion_buddhism\"\n 45       , \"<http://knowledgebooks.com/schema/topic/religion/buddhism>\")\n 46     , ( \"religion_islam\"\n 47       , \"<http://knowledgebooks.com/schema/topic/religion/islam>\")\n 48     , ( \"religion_christianity\"\n 49       , \"<http://knowledgebooks.com/schema/topic/religion/christianity>\")\n 50     , ( \"religion_hinduism\"\n 51       , \"<http://knowledgebooks.com/schema/topic/religion/hinduism>\")\n 52     , ( \"religion_judaism\"\n 53       , \"<http://knowledgebooks.com/schema/topic/religion/judaism>\")\n 54     , (\"chemistry\", \"<http://knowledgebooks.com/schema/topic/chemistry>\")\n 55     , (\"computers\", \"<http://knowledgebooks.com/schema/topic/computers>\")\n 56     , (\"computers_ai\", \"<http://knowledgebooks.com/schema/topic/computers/ai>\")\n 57     , ( \"computers_ai_datamining\"\n 58       , \"<http://knowledgebooks.com/schema/topic/computers/ai/datamining>\")\n 59     , ( \"computers_ai_learning\"\n 60       , \"<http://knowledgebooks.com/schema/topic/computers/ai/learning>\")\n 61     , ( \"computers_ai_nlp\"\n 62       , \"<http://knowledgebooks.com/schema/topic/computers/ai/nlp>\")\n 63     , ( \"computers_ai_search\"\n 64       , \"<http://knowledgebooks.com/schema/topic/computers/ai/search>\")\n 65     , ( \"computers_ai_textmining\"\n 66       , \"<http://knowledgebooks.com/schema/topic/computers/ai/textmining>\")\n 67     , ( \"computers/programming\"\n 68       , \"<http://knowledgebooks.com/schema/topic/computers/programming>\")\n 69     , ( \"computers_microsoft\"\n 70       , \"<http://knowledgebooks.com/schema/topic/computers/microsoft>\")\n 71     , ( \"computers/programming/ruby\"\n 72       , \"<http://knowledgebooks.com/schema/topic/computers/programming/ruby>\")\n 73     , ( \"computers/programming/lisp\"\n 74       , \"<http://knowledgebooks.com/schema/topic/computers/programming/lisp>\")\n 75     , (\"health\", \"<http://knowledgebooks.com/schema/topic/health>\")\n 76     , ( \"health_exercise\"\n 77       , \"<http://knowledgebooks.com/schema/topic/health/exercise>\")\n 78     , ( \"health_nutrition\"\n 79       , \"<http://knowledgebooks.com/schema/topic/health/nutrition>\")\n 80     , (\"mathematics\", \"<http://knowledgebooks.com/schema/topic/mathematics>\")\n 81     , (\"news_music\", \"<http://knowledgebooks.com/schema/topic/music>\")\n 82     , (\"news_physics\", \"<http://knowledgebooks.com/schema/topic/physics>\")\n 83     , (\"news_sports\", \"<http://knowledgebooks.com/schema/topic/sports>\")\n 84     ]\n 85 \n 86 uri_from_category :: [Char] -> [Char]\n 87 uri_from_category key =\n 88   fromMaybe (\"\\\"\" ++ key ++ \"\\\"\") $ M.lookup key category_to_uri_map\n 89 \n 90 textToTriples :: FilePath -> [Char] -> IO [Char]\n 91 textToTriples file_path meta_file_path = do\n 92   word_tokens <- filePathToWordTokens file_path\n 93   contents <- filePathToString file_path\n 94   putStrLn $ \"** contents:\\n\" ++ contents ++ \"\\n\"\n 95   meta_data <- readMetaFile meta_file_path\n 96   let people = peopleNames word_tokens\n 97   let companies = companyNames word_tokens\n 98   let countries = countryNames word_tokens\n 99   let cities = cityNames word_tokens\n100   let broadcast_networks = broadcastNetworkNames word_tokens\n101   let political_parties = politicalPartyNames word_tokens\n102   let trade_unions = tradeUnionNames word_tokens\n103   let universities = universityNames word_tokens\n104   let a_summary = summarizeS contents\n105   let the_categories = bestCategories word_tokens\n106   let filtered_categories =\n107         map (uri_from_category . fst) $\n108         filter (\\(name, value) -> value > 0.3) the_categories\n109   putStrLn \"\\nfiltered_categories:\"\n110   print filtered_categories\n111   --putStrLn \"a_summary:\"\n112   --print a_summary\n113   --print $ summarize contents\n114 \n115   let summary_triples =\n116         generate_triple\n117           (uri meta_data)\n118           \"<http://knowledgebooks.com/schema/summaryOf>\" $\n119         \"\\\"\" ++ a_summary ++ \"\\\"\"\n120   let category_triples =\n121         concat\n122           [ generate_triple\n123             (uri meta_data)\n124             \"<http://knowledgebooks.com/schema/news/category/>\"\n125             cat\n126           | cat <- filtered_categories\n127           ]\n128   let people_triples1 =\n129         concat\n130           [ generate_triple\n131             (uri meta_data)\n132             \"<http://knowledgebooks.com/schema/containsPersonDbPediaLink>\"\n133             (snd pair)\n134           | pair <- people\n135           ]\n136   let people_triples2 =\n137         concat\n138           [ generate_triple\n139             (snd pair)\n140             \"<http://knowledgebooks.com/schema/aboutPersonName>\"\n141             (make_literal (fst pair))\n142           | pair <- people\n143           ]\n144   let company_triples1 =\n145         concat\n146           [ generate_triple\n147             (uri meta_data)\n148             \"<http://knowledgebooks.com/schema/containsCompanyDbPediaLink>\"\n149             (snd pair)\n150           | pair <- companies\n151           ]\n152   let company_triples2 =\n153         concat\n154           [ generate_triple\n155             (snd pair)\n156             \"<http://knowledgebooks.com/schema/aboutCompanyName>\"\n157             (make_literal (fst pair))\n158           | pair <- companies\n159           ]\n160   let country_triples1 =\n161         concat\n162           [ generate_triple\n163             (uri meta_data)\n164             \"<http://knowledgebooks.com/schema/containsCountryDbPediaLink>\"\n165             (snd pair)\n166           | pair <- countries\n167           ]\n168   let country_triples2 =\n169         concat\n170           [ generate_triple\n171             (snd pair)\n172             \"<http://knowledgebooks.com/schema/aboutCountryName>\"\n173             (make_literal (fst pair))\n174           | pair <- countries\n175           ]\n176   let city_triples1 =\n177         concat\n178           [ generate_triple\n179             (uri meta_data)\n180             \"<http://knowledgebooks.com/schema/containsCityDbPediaLink>\"\n181             (snd pair)\n182           | pair <- cities\n183           ]\n184   let city_triples2 =\n185         concat\n186           [ generate_triple\n187             (snd pair)\n188             \"<http://knowledgebooks.com/schema/aboutCityName>\"\n189             (make_literal (fst pair))\n190           | pair <- cities\n191           ]\n192   let bnetworks_triples1 =\n193         concat\n194           [ generate_triple\n195             (uri meta_data)\n196             \"<http://knowledgebooks.com/schema/containsBroadCastDbPediaLink>\"\n197             (snd pair)\n198           | pair <- broadcast_networks\n199           ]\n200   let bnetworks_triples2 =\n201         concat\n202           [ generate_triple\n203             (snd pair)\n204             \"<http://knowledgebooks.com/schema/aboutBroadCastName>\"\n205             (make_literal (fst pair))\n206           | pair <- broadcast_networks\n207           ]\n208   let pparties_triples1 =\n209         concat\n210           [ generate_triple\n211             (uri meta_data)\n212             \"<http://knowledgebooks.com/schema/containsPoliticalPartyDbPediaLink>\"\n213             (snd pair)\n214           | pair <- political_parties\n215           ]\n216   let pparties_triples2 =\n217         concat\n218           [ generate_triple\n219             (snd pair)\n220             \"<http://knowledgebooks.com/schema/aboutPoliticalPartyName>\"\n221             (make_literal (fst pair))\n222           | pair <- political_parties\n223           ]\n224   let unions_triples1 =\n225         concat\n226           [ generate_triple\n227             (uri meta_data)\n228             \"<http://knowledgebooks.com/schema/containsTradeUnionDbPediaLink>\"\n229             (snd pair)\n230           | pair <- trade_unions\n231           ]\n232   let unions_triples2 =\n233         concat\n234           [ generate_triple\n235             (snd pair)\n236             \"<http://knowledgebooks.com/schema/aboutTradeUnionName>\"\n237             (make_literal (fst pair))\n238           | pair <- trade_unions\n239           ]\n240   let universities_triples1 =\n241         concat\n242           [ generate_triple\n243             (uri meta_data)\n244             \"<http://knowledgebooks.com/schema/containsUniversityDbPediaLink>\"\n245             (snd pair)\n246           | pair <- universities\n247           ]\n248   let universities_triples2 =\n249         concat\n250           [ generate_triple\n251             (snd pair)\n252             \"<http://knowledgebooks.com/schema/aboutTradeUnionName>\"\n253             (make_literal (fst pair))\n254           | pair <- universities\n255           ]\n256   return $\n257     concat\n258       [ people_triples1\n259       , people_triples2\n260       , company_triples1\n261       , company_triples2\n262       , country_triples1\n263       , country_triples2\n264       , city_triples1\n265       , city_triples2\n266       , bnetworks_triples1\n267       , bnetworks_triples2\n268       , pparties_triples1\n269       , pparties_triples2\n270       , unions_triples1\n271       , unions_triples2\n272       , universities_triples1\n273       , universities_triples2\n274       , category_triples\n275       , summary_triples\n276       ]\n

\n\n

The code in this file could be shortened but having repetitive code for each entity type hopefully makes it easier for you to understand how it works:

This code processes text from a given file and generates RDF triples (subject-predicate-object statements) based on the extracted information.

Key Functionality

\ncategory_to_uri_map: A map defining the correspondence between categories and their URIs.\n
\nuri_from_category: Retrieves the URI associated with a category, or returns the category itself in quotes if not found in the map.\n
\n
\ntextToTriples:\n
\n
\n
- \nTakes file paths for the text and metadata files.\n
- \nExtracts various entities (people, companies, countries, etc.) and categories from the text.\n
- \n
  \nGenerates RDF triples representing:\n
  \n
  \n
  - \nSummary of the text\n
  - \nCategories associated with the text\n
  - \nLinks between the text’s URI and identified entities (people, companies, etc.)\n
  - \nAdditional information about each identified entity (e.g., name)\n
  \n
  \n
- \nReturns a concatenated string of all generated triples.\n
\n
\n

Pattern

The code repeatedly follows this pattern for different entity types:

\nIdentify entities of a certain type (e.g., peopleNames).\n
\nGenerate triples linking the text’s URI to the entity’s URI.\n
\nGenerate triples providing additional information about the entity itself.\n

Purpose

This code is designed for knowledge extraction and representation. It aims to transform unstructured text into structured RDF data, making it suitable for semantic web applications or knowledge graphs.

Note:

\nThe code relies on external modules (Categorize, Entities, FileUtils, Summarize) for specific functionalities like categorization, entity recognition, file handling, and summarization.\n
\nThe quality of the generated triples will depend on the accuracy of these external modules.\n

Utility Code for Generating Cypher Input Data for Neo4J

Now we will generate Neo4J Cypher data. In order to keep the implementation simple, both the RDF and Cypher generation code starts with raw text and performs the NLP analysis to find entities. This example could be refactored to perform the NLP analysis just one time but in practice you will likely be working with either RDF or NEO4J and so you will probably extract just the code you need from this example (i.e., either the RDF or Cypher generation code).

Before we look at the code, let’s start with a few lines of generated Neo4J Cypher import data:

\n\n

 1 CREATE (newsshop_com_june_z902_html_news)-[:ContainsCompanyDbPediaLink]->(Wall_Stree\\\n 2 t_Journal)\n 3 CREATE (Canada:Entity {name:\"Canada\", uri:\"<http://dbpedia.org/resource/Canada>\"})\n 4 CREATE (newsshop_com_june_z902_html_news)-[:ContainsCountryDbPediaLink]->(Canada)\n 5 CREATE (summary_of_abcnews_go_com_US_violent_long_lasting_tornadoes_threaten_oklahom\\\n 6 a_texas_storyid63146361:Summary {name:\"summary_of_abcnews_go_com_US_violent_long_las\n 7 ting_tornadoes_threaten_oklahoma_texas_storyid63146361\", uri:\"<https://abcnews.go.co\n 8 m/US/violent-long-lasting-tornadoes-threaten-oklahoma-texas/story?id=63146361>\", sum\n 9 mary:\"Part of the system that delivered severe weather to the central U.S. over the \n10 weekend is moving into the Northeast today, producing strong to severe storms -- dam\n11 aging winds, hail or isolated tornadoes can't be ruled out. Severe weather is foreca\n12 st to continue on Tuesday, with the western storm moving east into the Midwest and p\n13 arts of the mid-Mississippi Valley.\"})\n

\n\n

The following listing shows the file src/sw/GenNeo4jCypher.hs. This code is very similar to the code for generating RDF in the last section. The same notes for adding your own new entity notes in the last section are also relevant here.

Notice that we import in line 29 the map category_to_uri_map that was defined in the last section. The function neo4j_category_node_defs defined in lines 35 to 43 creates category graph nodes for each category in the map category_to_uri_map. These nodes will be referenced by graph nodes created in the functions create_neo4j_node, create_neo4j_lin, create_summary_node, and create_entity_node. The top level function is textToCypher that is similar to the function textToTriples in the last section.

  1 {-# LANGUAGE OverloadedStrings #-}\n  2 \n  3 module GenNeo4jCypher\n  4   ( textToCypher\n  5   , neo4j_category_node_defs\n  6   ) where\n  7 \n  8 import Categorize (bestCategories)\n  9 import Data.List (isInfixOf)\n 10 import Data.Char (toLower)\n 11 import Data.String.Utils (replace)\n 12 import Entities\n 13   ( broadcastNetworkNames\n 14   , cityNames\n 15   , companyNames\n 16   , countryNames\n 17   , peopleNames\n 18   , politicalPartyNames\n 19   , tradeUnionNames\n 20   , universityNames\n 21   )\n 22 import FileUtils\n 23   ( MyMeta\n 24   , filePathToString\n 25   , filePathToWordTokens\n 26   , readMetaFile\n 27   , uri\n 28   )\n 29 import GenTriples (category_to_uri_map)\n 30 import Summarize (summarize, summarizeS)\n 31 \n 32 import qualified Data.Map as M\n 33 import Data.Maybe (fromMaybe)\n 34 import Database.SQLite.Simple\n 35 \n 36 -- for debug:\n 37 import Data.Typeable (typeOf)\n 38 \n 39 neo4j_category_node_defs :: [Char]\n 40 neo4j_category_node_defs =\n 41   replace\n 42     \"/\"\n 43     \"_\"\n 44     $ concat\n 45     [ \"CREATE (\" ++ c ++ \":CategoryType {name:\\\"\" ++ c ++ \"\\\"})\\n\"\n 46     | c <- M.keys category_to_uri_map\n 47     ]\n 48 \n 49 uri_from_category :: p -> p\n 50 uri_from_category s = s -- might want the full version from GenTriples\n 51 \n 52 repl :: Char -> Char\n 53 repl '-' = '_'\n 54 repl '/' = '_'\n 55 repl '.' = '_'\n 56 repl c = c\n 57 \n 58 filterChars :: [Char] -> [Char]\n 59 filterChars = filter (\\c -> c /= '?' && c /= '=' && c /= '<' && c /= '>')\n 60 \n 61 create_neo4j_node :: [Char] -> ([Char], [Char])\n 62 create_neo4j_node uri =\n 63   let name =\n 64         (map repl (filterChars\n 65                     (replace \"https://\" \"\" (replace \"http://\" \"\" uri)))) ++\n 66                     \"_\" ++\n 67                     (map toLower node_type)\n 68       node_type =\n 69         if isInfixOf \"dbpedia\" uri\n 70           then \"DbPedia\"\n 71           else \"News\"\n 72       new_node =\n 73         \"CREATE (\" ++\n 74         name ++ \":\" ++\n 75         node_type ++ \" {name:\\\"\" ++ (replace \" \" \"_\" name) ++\n 76         \"\\\", uri:\\\"\" ++ uri ++ \"\\\"})\\n\"\n 77    in (name, new_node)\n 78 \n 79 create_neo4j_link :: [Char] -> [Char] -> [Char] -> [Char]\n 80 create_neo4j_link node1 linkName node2 =\n 81   \"CREATE (\" ++ node1 ++ \")-[:\" ++ linkName ++ \"]->(\" ++ node2 ++ \")\\n\"\n 82 \n 83 create_summary_node :: [Char] -> [Char] -> [Char]\n 84 create_summary_node uri summary =\n 85   let name =\n 86         \"summary_of_\" ++\n 87         (map repl $\n 88          filterChars (replace \"https://\" \"\" (replace \"http://\" \"\" uri)))\n 89       s1 = \"CREATE (\" ++ name ++ \":Summary {name:\\\"\" ++ name ++ \"\\\", uri:\\\"\"\n 90       s2 = uri ++ \"\\\", summary:\\\"\" ++ summary ++ \"\\\"})\\n\"\n 91    in s1 ++ s2\n 92 \n 93 create_entity_node :: ([Char], [Char]) -> [Char]\n 94 create_entity_node entity_pair = \n 95   \"CREATE (\" ++ (replace \" \" \"_\" (fst entity_pair)) ++ \n 96   \":Entity {name:\\\"\" ++ (fst entity_pair) ++ \"\\\", uri:\\\"\" ++\n 97   (snd entity_pair) ++ \"\\\"})\\n\"\n 98 \n 99 create_contains_entity :: [Char] -> [Char] -> ([Char], [Char]) -> [Char]\n100 create_contains_entity relation_name source_uri entity_pair =\n101   let new_person_node = create_entity_node entity_pair\n102       new_link = create_neo4j_link source_uri\n103                    relation_name\n104                    (replace \" \" \"_\" (fst entity_pair))\n105   in\n106     (new_person_node ++ new_link)\n107 \n108 entity_node_helper :: [Char] -> [Char] -> [([Char], [Char])] -> [Char]\n109 entity_node_helper relation_name node_name entity_list =\n110   concat [create_contains_entity\n111            relation_name node_name entity | entity <- entity_list]\n112 \n113 textToCypher :: FilePath -> [Char] -> IO [Char]\n114 textToCypher file_path meta_file_path = do\n115   let prelude_nodes = neo4j_category_node_defs\n116   putStrLn \"+++++++++++++++++ prelude node defs:\"\n117   print prelude_nodes\n118   word_tokens <- filePathToWordTokens file_path\n119   contents <- filePathToString file_path\n120   putStrLn $ \"** contents:\\n\" ++ contents ++ \"\\n\"\n121   meta_data <- readMetaFile meta_file_path\n122   putStrLn \"++ meta_data:\"\n123   print meta_data\n124   let people = peopleNames word_tokens\n125   let companies = companyNames word_tokens\n126   putStrLn \"^^^^ companies:\"\n127   print companies\n128   let countries = countryNames word_tokens\n129   let cities = cityNames word_tokens\n130   let broadcast_networks = broadcastNetworkNames word_tokens\n131   let political_parties = politicalPartyNames word_tokens\n132   let trade_unions = tradeUnionNames word_tokens\n133   let universities = universityNames word_tokens\n134   let a_summary = summarizeS contents\n135   let the_categories = bestCategories word_tokens\n136   let filtered_categories =\n137         map (uri_from_category . fst) $\n138         filter (\\(name, value) -> value > 0.3) the_categories\n139   putStrLn \"\\nfiltered_categories:\"\n140   print filtered_categories\n141   let (node1_name, node1) = create_neo4j_node (uri meta_data)\n142   let summary1 = create_summary_node (uri meta_data) a_summary\n143   let category1 =\n144         concat\n145           [ create_neo4j_link node1_name \"Category\" cat\n146           | cat <- filtered_categories\n147           ]\n148   let pp = entity_node_helper \"ContainsPersonDbPediaLink\" node1_name people\n149   let cmpny = entity_node_helper \"ContainsCompanyDbPediaLink\" node1_name companies\n150   let cntry = entity_node_helper \"ContainsCountryDbPediaLink\" node1_name countries\n151   let citys = entity_node_helper \"ContainsCityDbPediaLink\" node1_name cities\n152   let bnet = entity_node_helper \"ContainsBroadcastNetworkDbPediaLink\"\n153                                 node1_name broadcast_networks\n154   let ppart = entity_node_helper \"ContainsPoliticalPartyDbPediaLink\"\n155                                 node1_name political_parties\n156   let tunion = entity_node_helper \"ContainsTradeUnionDbPediaLink\"\n157                                   node1_name trade_unions\n158   let uni = entity_node_helper \"ContainsUniversityDbPediaLink\"\n159                                node1_name universities\n160   return $ concat [node1, summary1, category1, pp, cmpny, cntry, citys, bnet,\n161                    ppart, tunion, uni]\n

\n\n

This code generates Cypher queries to create nodes and relationships in a Neo4j graph database based on extracted information from text.

Core Functionality:

\nneo4j_category_node_defs: Defines Cypher statements to create nodes for predefined categories.\n
\nuri_from_category: Placeholder, potentially for full URI mapping (not used in this code).\n
\ncreate_neo4j_node: Creates a Cypher statement to create a node representing either a DbPedia entity or a News article, based on the URI.\n
\ncreate_neo4j_link: Creates a Cypher statement to create a relationship between two nodes.\n
\ncreate_summary_node: Creates a Cypher statement to create a node representing a summary of the text.\n
\ncreate_entity_node: Creates a Cypher statement to create a node representing an entity.\n
\ncreate_contains_entity: Creates Cypher statements to create an entity node and link it to a source node with a specified relationship.\n
\nentity_node_helper: Generates Cypher statements for creating entity nodes and relationships for a list of entities.\n
\n
\ntextToCypher:\n
\n
\n
- \nProcesses text from a file and its metadata.\n
- \nExtracts various entities and categories from the text.\n
- \n
  \nGenerates Cypher statements to:\n
  \n
  \n
  - \nCreate nodes for the text itself, its summary, and identified categories.\n
  - \nCreate nodes and relationships for entities (people, companies, etc.) mentioned in the text.\n
  \n
  \n
- \nReturns a concatenated string of all generated Cypher statements.\n
\n
\n

Purpose:

This code is designed to transform text into a structured representation within a Neo4j graph database. This allows for querying and analyzing relationships between entities and categories extracted from the text.

Because the top level function is textToCypher returns a string wrapped in a monad, it is possible to add “debug”“ print statements in textToCypher. I left many such debug statements in the example code to help you understand the data that is being operated on. I leave it as an exercise to remove these print statements if you use this code in your own projects and no longer need to see the debug output.

Top Level API Code for Handling Knowledge Graph Data Generation

So far we have looked at processing command line arguments and processing individual input files. Now we look at higher level utility APIs for processing an entire directory of input files. The following listing shows the file API.hs that contains the two top level helper functions we saw in app/Main.hs.

The functions processFilesToRdf and processFilesToNeo4j both have the function type signature FilePath->FilePath->IO() and are very similar except for calling different helper functions to generate RDF triples or Cypher input graph data:

 1 module Apis\n 2   ( processFilesToRdf\n 3   , processFilesToNeo4j\n 4   ) where\n 5 \n 6 import FileUtils\n 7 import GenNeo4jCypher\n 8 import GenTriples (textToTriples)\n 9 \n10 import qualified Database.SQLite.Simple as SQL\n11 \n12 import Control.Monad (mapM)\n13 import Data.String.Utils (replace)\n14 import System.Directory (getDirectoryContents)\n15 \n16 import Data.Typeable (typeOf)\n17 \n18 processFilesToRdf :: FilePath -> FilePath -> IO ()\n19 processFilesToRdf dirPath outputRdfFilePath = do\n20   files <- getDirectoryContents dirPath :: IO [FilePath]\n21   let filtered_files = filter isTextFile files\n22   let full_paths = [dirPath ++ \"/\" ++ fn | fn <- filtered_files]\n23   putStrLn \"full_paths:\"\n24   print full_paths\n25   let r =\n26         [textToTriples fp1 (replace \".txt\" \".meta\" fp1)\n27         |\n28         fp1 <- full_paths] :: [IO [Char]]\n29   tripleL <-\n30     mapM (\\fp -> textToTriples fp (replace \".txt\" \".meta\" fp)) full_paths\n31   let tripleS = concat tripleL\n32   putStrLn tripleS\n33   writeFile outputRdfFilePath tripleS\n34 \n35 processFilesToNeo4j :: FilePath -> FilePath -> IO ()\n36 processFilesToNeo4j dirPath outputRdfFilePath = do\n37   files <- getDirectoryContents dirPath :: IO [FilePath]\n38   let filtered_files = filter isTextFile files\n39   let full_paths = [dirPath ++ \"/\" ++ fn | fn <- filtered_files]\n40   putStrLn \"full_paths:\"\n41   print full_paths\n42   let prelude_node_defs = neo4j_category_node_defs\n43   putStrLn\n44     (\"+++++  type of prelude_node_defs is: \" ++\n45      (show (typeOf prelude_node_defs)))\n46   print prelude_node_defs\n47   cypher_dataL <-\n48     mapM (\\fp -> textToCypher fp (replace \".txt\" \".meta\" fp)) full_paths\n49   let cypher_dataS = concat cypher_dataL\n50   putStrLn cypher_dataS\n51   writeFile outputRdfFilePath $ prelude_node_defs ++ cypher_dataS\n

\n\n

Since both of these functions return IO monads, I could add “debug” print statements that should be helpful in understanding the data being operated on.

The code defines two functions for processing text files in a directory:

\n
processFilesToRdf: Processes text files and their corresponding metadata files (with .meta extension) in a given directory. It converts the content into RDF triples using textToTriples and writes the concatenated triples to an output RDF file.
\n
\n
processFilesToNeo4j: Processes text files and metadata files to generate Cypher statements for Neo4j. It uses textToCypher to create Cypher data from file content, combines it with predefined Neo4j category node definitions, and writes the result to an output file.
\n

Key Points

\n
File Handling: It utilizes getDirectoryContents for file listing, filter for selecting text files, and writeFile for output.
\n
\n
Data Transformation: textToTriples and textToCypher are functions that convert text content into RDF triples and Cypher statements, respectively.
\n
\n
Metadata Handling: It expects metadata files with the same base name as the text files but with a .meta extension.
\n
\n
Output: The generated RDF triples or Cypher statements are written to specified output files.
\n
\n
neo4j_category_node_defs: A variable holding predefined Cypher node definitions for Neo4j categories.
\n

This code relies on external modules like FileUtils, GenNeo4jCypher, GenTriples, and Database.SQLite.Simple for specific functionalities.

Wrap Up for Automating the Creation of Knowledge Graphs

The code in this chapter will provide you with a good start for creating both test knowledge graphs and for generating data for production. In practice, generated data should be reviewed before use and additional data manually generated as needed. It is good practice to document required manual changes because this documentation can be used in the requirements for updating the code in this chapter to more closely match your knowledge graph requirements.

Hybrid Haskell and Python Natural Language Processing

Here we will write a Haskell client for using a Natural Language Processing (NLP) server written in Python. There is some common material in this chapter and the next chapter Hybrid Haskell and Python For Coreference Resolution because I wanted both chapters to be self contained.

Example Use of the Haskell NLP Client

Before learning how to use the Python NLP server code and to understand the code for the Haskell client code, let’s look at an example of running the client code so you understand the type of processing that we are performing:

1 $ stack build --fast --exec HybridHaskellPythonNlp-exe\n2 Enter text (all on one line)\n3 John Smith went to Mexico to see the Pepsi plant\n4 response from NLP server:\n5 NlpResponse {entities = [\"John Smith/PERSON\",\"Mexico/GPE\",\"Pepsi/ORG\"],\n6              tokens = [\"John\",\"Smith\",\"went\",\"to\",\"Mexico\",\"to\",\"see\",\n7                        \"the\",\"Pepsi\",\"plant\"]}\n8 Enter text (all on one line)\n

\n\n

Notice on line 5 that each of the three entities is tagged with the entity type. GPE is the tag for a country and the tag ORG can refer to an entity that is a company or a non-profit organization.

There is some overlap in functionality between the Python SpaCy NLP library and my pure Haskell code in the NLP Tools chapter. SpaCy has the advantage of using state of the art deep learning models.

Setting up the Python NLP Server

I assume that you have some familiarity with using Python. If not, you will still be able to follow these directions assuming that you have the utilities pip, and python installed. I recommend installing Python and Pip using Anaconda.

The server code is in the subdirectory HybridHaskellPythonNlp/python_spacy_nlp_server where you will work when performing a one time initialization. After the server is installed you can then run it from the command line from any directory on your laptop.

I recommend that you use virtual Python environments when using Python applications to separate the dependencies required for each application or development project. Here I assume that you are running in a Python version 3.6 (or higher) version environment. First install the dependencies:

pip install -U spacy\npython -m spacy download en\npip install falcon\n

\n\n

Then change directory to the subdirectory HybridHaskellPythonNlp/python_spacy_nlp_server and install the NLP server:

cd HybridHaskellPythonNlp/python_spacy_nlp_server\npython setup.py install\n

\n\n

Once you install the server, you can run it from any directory on your laptop or server using:

spacynlpserver\n

\n\n

I use deep learning models written in Python using TensorFlow or PyTorch in applications I write in Haskell or Common Lisp. While it is possible to directly embed models in Haskell and Common Lisp, I find it much easier and developer friendly to wrap deep learning models I use a REST services as I have done here. Often deep learning models only require about a gigabyte of memory and using pre-trained models has lightweight CPU resource needs so while I am developing on my laptop I might have two or three models running and available as wrapped REST services. For production, I configure both the Python services and my Haskell and Common Lisp applications to start automatically on system startup.

This is not a Python programming book and I will not discuss the simple Python wrapping code but if you are also a Python developer you can easily read and understand the code.

Understanding the Haskell NLP Client Code

The Python server returns JSON file. We saw earlier the use of the Haskell aeson library for parsing JSON data stored as a string into Haskell native data. We also used the wreq library to access remote web services. We use both of these libraries here:

{-# LANGUAGE OverloadedStrings #-}\n{-# LANGUAGE DeriveDataTypeable #-}\n\n-- reference: http://www.serpentine.com/wreq/tutorial.html\nmodule NlpWebClient\n  ( nlpClient, NlpResponse\n  ) where\n\nimport Control.Lens\nimport Data.ByteString.Lazy.Char8 (unpack)\nimport Data.Maybe (fromJust)\nimport Network.URI.Encode as E -- encode is also in Data.Aeson\nimport Network.Wreq\n\nimport Text.JSON.Generic\n\ndata NlpResponse = NlpResponse {entities::[String], tokens::[String]} deriving (Show\\\n, Data, Typeable)\n\nbase_url = \"http://127.0.0.1:8008?text=\"\n\nnlpClient :: [Char] -> IO NlpResponse\nnlpClient query = do\n  putStrLn $ \"\\n\\n***  Processing \" ++ query\n  r <- get $ base_url ++ (E.encode query) ++ \"&no_detail=1\"\n  let ret = (decodeJSON (unpack (fromJust (r ^? responseBody)))) :: NlpResponse\n  return ret\n

\n\n

This Haskell code snippet defines a simple web client (nlpClient) that interacts with an NLP (Natural Language Processing) service.

\nIt sends a query text to the NLP service.\n
\nIt receives a JSON response containing entities and tokens extracted from the query.\n
\nIt parses the JSON response and returns the structured data.\n

Code Explanation

1. Language Extensions

\nOverloadedStrings: Allows using string literals (\"like this\") as both String and Text types, making string handling more convenient.\n
\nDeriveDataTypeable: Automatically generates instances of the Data and Typeable classes for the NlpResponse data type. This enables easy serialization/deserialization and runtime type checking.\n

2. Imports

\nControl.Lens: Used for concisely accessing nested data structures (like r ^? responseBody).\n
\nData.ByteString.Lazy.Char8: Handles lazy ByteStrings, which are efficient for processing large responses.\n
\nData.Maybe: Provides the fromJust function to safely extract values from Maybe types.\n
\nNetwork.URI.Encode: Used for URL-encoding the query text.\n
\nNetwork.Wreq: A simple HTTP client library for making requests.\n
\nText.JSON.Generic: For generic JSON parsing.\n

3. NlpResponse Data Type

\nRepresents the structured data returned by the NLP service.\n
\n
\nHas two fields:\n
\n
\n
- \nentities: A list of strings representing the identified entities.\n
- \ntokens: A list of strings representing the tokens in the query.\n
\n
\n

4. base_url

\nThe base URL of the NLP service.\n
\nAssumes the service runs locally (127.0.0.1) on port 8008.\n
\nTakes the query text as a parameter (?text=).\n
\nAppends &no_detail=1 to request a simplified response.\n

5. nlpClient Function

\nTakes a query text ([Char]) as input.\n
\nPrints a log message indicating the query being processed.\n
\n
\nConstructs the full URL by:\n
\n
\n
- \nAppending the base URL.\n
- \nURL-encoding the query using E.encode.\n
- \nAdding the &no_detail=1 parameter.\n
\n
\n
\nMakes an HTTP GET request to the constructed URL using get.\n
\nExtracts the response body from the result using r ^? responseBody.\n
\nDecodes the JSON response using decodeJSON and unpack.\n
\nReturns the parsed NlpResponse.\n

This code provides a basic way to interact with an NLP service from Haskell. It sends queries, handles the response, and extracts relevant information in a structured format.

The main command line program for using the client library:

module Main where\n\nimport NlpWebClient\n    \nmain :: IO ()\nmain = do\n  putStrLn \"Enter text (all on one line)\"\n  s <- getLine\n  response <- (nlpClient s) :: IO NlpResponse\n  putStr \"response from NLP server:\\n\"\n  putStrLn $ show response\n  main\n

\n\n

This code snippet defines a simple interactive application that:

\nPrompts the user to enter a line of text.\n
\nSends the entered text to an NLP (Natural Language Processing) service using the nlpClient function from the NlpWebClient module.\n
\nPrints the response received from the NLP service.\n
\nRepeats the process (allowing the user to enter another query).\n

Explanation

\n
\n
Imports:
\n
\n
\n
- \nNlpWebClient: Imports the nlpClient function and the NlpResponse data type, presumably defined in another module.\n
\n
\n
\n
\n
main Function:
\n
\n
\n
- \nputStrLn \"Enter text (all on one line)\": Prints a prompt to the user.\n
- \ns <- getLine: Reads a line of input from the user and stores it in the s variable.\n
- \n
  \nresponse <- (nlpClient s) :: IO NlpResponse:\n
  \n
  \n
  - \nCalls the nlpClient function with the user’s input (s).\n
  - \nThe :: IO NlpResponse type annotation ensures that the result of nlpClient s is treated as an IO action that produces an NlpResponse.\n
  - \nThe result of this action (the NlpResponse value) is bound to the response variable.\n
  \n
  \n
- \nputStr \"response from NLP server:\\n\": Prints a label for the response.\n
- \n
  \nputStrLn $ show response:\n
  \n
  \n
  - \nUses show to convert the NlpResponse value to a string representation.\n
  - \nPrints the string representation of the response.\n
  \n
  \n
- \nmain: Recursively calls the main function, creating an infinite loop that allows the user to enter multiple queries.\n
\n
\n

Key Points

\nInteraction: The code interacts with the user through the console, taking input and displaying output.\n
\nNLP Processing: It relies on the nlpClient function from the NlpWebClient module to communicate with an NLP service and process the user’s input.\n
\nInfinite Loop: The recursive call to main at the end creates a continuous loop, allowing the user to enter multiple queries until they manually terminate the program.\n

Wrap Up for Using the Python SpaCy NLP Service

The example in this chapter shows a technique that I often use for using libraries and frameworks that are not written in Haskell: wrap the service implemented in another programming language is a REST web service. While it is possible to use a foreign function interface (FFI) to call out to code written in other languages I find for my own work that I prefer calling out to a separate service especially when I run other services on remote servers so I do not need to run them on my development laptop. For production it is also useful to be able to easily scale horizontally across servers.

Hybrid Haskell and Python For Coreference Resolution

Here we will write a Haskell client for using a server written in Python that performs coreference resolution (more on this later). There is some common material in this chapter and the last chapter Hybrid Haskell and Python Natural Language Processing because I wanted both chapters to be self contained. The code for this chapter can be found in the subdirectory HybridHaskellPythonCorefAnaphoraResolution.

Coreference resolution is also called anaphora resolution and is the process for replacing pronouns in text with the original nouns, proper nouns, or noun phrases that the pronouns refer to.

Before discussing setting up the Python library for performing coreference analysis and the Haskell client, let’s run the client so you can see and understand anaphora resolution:

 $ stack build --fast --exec HybridHaskellPythonCorefAnaphoraResolution-exe\nEnter text (all on one line)\nJohn Smith drove a car. He liked it.\n\n\n***  Processing John%20Smith%20drove%20a%20car.%20He%20liked%20it.\nstatus code: 200\ncontent type: Just \"application/text\"\nresponse body: John Smith drove a car. John Smith liked a car.\nresponse from coreference server:   \"John Smith drove a car. John Smith liked a car.\"\nEnter text (all on one line)\n

\n\n

In this example notice that the words “He” and “it” in the second sentence are replaced by “John Smith” and “a car” which makes it easier to write information extraction applications.

Installing the Python Coreference Server

pip install spacy==2.1.0\npip install neuralcoref \npip install falcon\n

\n\n

As I write this chapter the neuralcoref model and library require a old version of spaCy.

If you want to instead use the latest version of spaCy then install neuralcoref from source:

pip install spacy\ngit clone https://github.com/huggingface/neuralcoref.git\ncd neuralcoref\npython setup.py install\npip install falcon\n

\n\n

After installing all dependencies, then change directory to the subdirectory python_coreference_anaphora_resolution_server and install the coref server:

cd python_coreference_anaphora_resolution_server\npython setup.py install\n

\n\n

Once you install the server, you can run it from any directory on your laptop or server using:

corefserver\n

\n\n

I use deep learning models written in Python using TensorFlow or PyTorch in applications I write in Haskell or Common Lisp. While it is possible to directly embed models in Haskell and Common Lisp, I find it much easier and developer friendly to wrap deep learning models I use as REST services as I have done here. Often deep learning models only require about a gigabyte of memory and using pre-trained models has lightweight CPU resource needs so while I am developing on my laptop I might have two or three models running and available as wrapped REST services. For production, I configure both the Python services and my Haskell and Common Lisp applications to start automatically on system startup.

This is not a Python programming book and I will not discuss the simple Python wrapping code but if you are also a Python developer you can easily read and understand the code.

Understanding the Haskell Coreference Client Code

The code for the library for fetching data from the Python service is in the subdirectory src in the file CorefWebClient.hs.

We will use techniques for accessing remote web services using the wreq library and using the lens library for accessing the response from the Python server. Here the response is plain text with pronouns replaced by the nouns that they represent. We don’t use the aeson library to parse JSON data as we did in the previous chapter.

{-# LANGUAGE OverloadedStrings #-}\n\n-- reference: http://www.serpentine.com/wreq/tutorial.html\nmodule CorefWebClient\n  ( corefClient\n  ) where\n\nimport Control.Lens\nimport Data.ByteString.Lazy.Char8 (unpack)\nimport Data.Maybe (fromJust)\nimport Network.URI.Encode (encode)\nimport Network.Wreq\n\nbase_url = \"http://127.0.0.1:8000?text=\"\n\ncorefClient :: [Char] -> IO [Char]\ncorefClient query = do\n  putStrLn $ \"\\n\\n***  Processing \" ++ (encode query)\n  r <- get $ base_url ++ (encode query) ++ \"&no_detail=1\"\n  putStrLn $ \"status code: \" ++ (show (r ^. responseStatus . statusCode))\n  putStrLn $ \"content type: \" ++ (show (r ^? responseHeader \"Content-Type\"))\n  putStrLn $ \"response body: \" ++ (unpack (fromJust (r ^? responseBody)))\n  return $ unpack (fromJust (r ^? responseBody))\n

\n\n

This code defines a function corefClient which acts as a simple web client to interact with a Coreference Resolution service.

Code Breakdown

\nbase_url: Stores the base URL of the Coreference Resolution service, assumed to be running locally on port 8000.\n
\n
\ncorefClient:\n
\n
\n
- \nTakes a text query as input.\n
- \nPrints the encoded query to the console.\n
- \nConstructs the full URL by appending the encoded query and &no_detail=1 to base_url.\n
- \nMakes an HTTP GET request to the constructed URL using get.\n
- \nPrints the status code, content type, and response body from the server’s response.\n
- \nReturns the response body as a string.\n
\n
\n

Key Points

\nOverloadedStrings language extension is used for convenient string handling.\n
\nControl.Lens provides tools for accessing nested data structures (like r ^. responseStatus . statusCode).\n
\nData.ByteString.Lazy.Char8 is used for efficient handling of the response body.\n
\nData.Maybe provides fromJust for safely extracting values from Maybe types.\n
\nNetwork.URI.Encode is used for URL-encoding the query.\n
\nNetwork.Wreq is a simple HTTP client library.\n

The code for the main application is in the subdirectory app in the file Main.hs.

module Main where\n\nimport CorefWebClient\n\nmain :: IO ()\nmain = do\n  putStrLn \"Enter text (all on one line)\"\n  s <- getLine\n  response <- corefClient s\n  putStr \"response from coreference server:\\t\"\n  putStrLn $ show response\n  main\n

\n\n

This Haskell code creates a basic interactive console application that:

\nPrompts the user to input a line of text.\n
\nSends the text to a Coreference Resolution service using the corefClient function.\n
\nPrints the response from the service.\n
\nRepeats the process, allowing continuous interaction.\n

Key Points

\ncorefClient (imported from CorefWebClient) is assumed to handle communication with the Coreference Resolution service.\n
\nThe code uses getLine for user input, putStrLn for output, and show to convert the response to a printable format.\n
\nThe recursive call to main creates an infinite loop, allowing the user to process multiple inputs until they manually terminate the program.\n

Wrap Up for Using the Python Coreference NLP Service

The example in this chapter is fairly simple but shows a technique that I often use for using libraries and frameworks that are not written in Haskell: wrap the service implemented in another programming language is a REST web service. While it is possible to use a foreign function interface (FFI) to call out to code written in other languages I find for my own work that I prefer calling out to a separate service, especially when I run other services on remote servers so I do not need to run them on my development laptop. For production it is also useful to be able to easily scale horizontally across servers.

Book Wrap Up

As I mentioned in the Preface, I had a slow start learning Haskell because I tried to learn too much at one time. In this book I have attempted to show you a subset of Haskell that is sufficient to write interesting programs - a gentle introduction.

Haskell beginners often dislike the large error listings from the compiler. The correct attitude is to recognize that these error messages are there to help you. That is easier said than done, but try to be happy when the compiler points out an error - in the long run I find using Haskell’s fussy compiler saves me time and lets me refactor code knowing that if I miss something in my refactoring the compiler will immediately let me know what needs to be fixed.

The other thing that I hope you learned working through this book is how effective repl based programming is. Most code I write, unless it is very trivial, starts its life in a GHCi repl. When you are working with someone else’s Haskell code it is similarly useful to have their code loaded in a repl as you read.

I have been programming professionally for forty years and I use many programming languages. Once I worked my way through early difficulties using Haskell it has become a favorite programming language. I hope that you enjoy Haskell development as much as I do.

Using Haskell for Modern AI Applications

I have been working professionally in the field of Artificial Intelligence since 1982, and it has been interesting to see how different programming languages have been used in AI research and deployed applications. In order, I have personally used: Common Lisp, Prolog, then back to Common Lisp, C++, Java, and Python for AI research and development. As I write this in September 2024, the use of Large Language Models (LLMs) has dominated AI research and development for the last three years. As you saw earlier in the chapter Using the OpenAI Large Language Model APIs in Haskell, it is straight forward to utilize LLMs in Haskell applications. With Haskell’s repl-based development coding efficiency, type safety, generally high performance, I think that Haskell is well situated to be an effective application and glue language for building modern AI applications.

Appendix A - Haskell Tools Setup

Here I discuss the stack build tool but as I update this book in September 2024, I now frequently just use Cabal to build and run projects. The README files for the examples in the GitHub repository for this book usually have directions for both stack and Cabal.

I recommend that if you are new to Haskell that you at least do a minimal installation of stack and work through the first chapter using an interactive REPL. After experimenting with the REPL then do please come back to Appendix A and install support for the editor of your choice (or an IDE) and hlint.

stack

I assume that you have the Haskell package manager stack installed. If you have not installed stack yet please follow these directions.

After installing stack and running it you will have a directory “.stack” in your home directory where stack will keep compiled libraries and configuration data. You will want to create a file “~/.stack/config.yaml” with contents similar to my stack configuration file:

\n\n

1 templates:\n2   params:\n3       author-email: markw@markwatson.com\n4       author-name: Mark Watson\n5       category: dev\n6       copyright: Copyright 2016-2024 Mark Watson. All rights reserved\n7       github-username: mark-watson\n

\n\n

Replace my name and email address with yours. You might also want to install the package manager Cabal and the “lint” program hlint:

\n\n

1 $ stack install cabal-install\n2 $ stack install hlint\n

\n\n

These installs might take a while so go outside for ten minutes and get some fresh air.

You should get in the habit of running hlint on your code and consider trying to remove all or at least most warnings. You can customize the types of warnings hlint shows: read the documentation for hlint.

Creating a New Stack Project

I have already created stack projects for the examples in this book. When you have worked through them, then please refer to the stack documentation for creating projects.

Emacs Setup

There are several good alternatives to using the Emacs editor but Emacs with haskell-mode is my favorite environment. There are instructions for adding haskell-mode to Emacs on the project home page on github. If you follow these instructions you will have syntax hi-liting and Emacs will understand Haskell indentation rules.

Do you want more of an IDE-like Development Environment?

I recommend and use the Intero Emacs package to get auto completions and real time syntax error warnings. Intero is designed to work with stack.

I add the following to the bottom of my .emacs file:

(add-hook ’haskell-mode-hook ’intero-mode)

and if Intero is too “heavy weight” for my current project, then I comment out the add-hook expression. Intero can increase the startup time for Emacs for editing Haskell files. That said, I almost always keep Intero enabled in my Emacs environment.

hlint

hlint is a wonderful tool for refining your knowledge and use of the Haskell language. After writing new code and checking that it works, then run hlint for suggestions on how to improve your code.

Install hlint using:

\n\n

1 stack install hlint\n

\n\n\n

\n\n","tocStructure",[438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512,513,514,515,516,517,518,519,520,521,522,523,524,525,526,527,528,529,530,531,532,533,534,535],{"_77":78,"_44":79,"_71":-5,"_80":81,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":87,"_71":-5,"_80":88,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":92,"_71":-5,"_80":93,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":97,"_71":-5,"_80":98,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":100,"_71":-5,"_80":101,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":103,"_71":-5,"_80":104,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":106,"_71":-5,"_80":107,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":109,"_71":-5,"_80":110,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":112,"_71":-5,"_80":113,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":115,"_71":-5,"_80":116,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":118,"_71":-5,"_80":119,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":121,"_71":-5,"_80":122,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":125,"_71":-5,"_80":126,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":129,"_71":-5,"_80":130,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":132,"_71":-5,"_80":133,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":135,"_71":-5,"_80":136,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":138,"_71":-5,"_80":139,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":141,"_71":-5,"_80":142,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":144,"_71":-5,"_80":145,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":147,"_71":-5,"_80":148,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":150,"_71":-5,"_80":151,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":153,"_71":-5,"_80":154,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":156,"_71":-5,"_80":157,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":159,"_71":-5,"_80":160,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":162,"_71":-5,"_80":163,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":165,"_71":-5,"_80":166,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":168,"_71":-5,"_80":169,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":171,"_71":-5,"_80":172,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":174,"_71":-5,"_80":175,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":177,"_71":-5,"_80":178,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":181,"_71":-5,"_80":182,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":184,"_71":-5,"_80":185,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":187,"_71":-5,"_80":188,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":190,"_71":-5,"_80":191,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":193,"_71":-5,"_80":194,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":196,"_71":-5,"_80":197,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":199,"_71":-5,"_80":200,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":202,"_71":-5,"_80":203,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":205,"_71":-5,"_80":206,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":208,"_71":-5,"_80":209,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":211,"_71":-5,"_80":212,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":214,"_71":-5,"_80":215,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":217,"_71":-5,"_80":218,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":220,"_71":-5,"_80":221,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":224,"_71":-5,"_80":225,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":228,"_71":-5,"_80":229,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":231,"_71":-5,"_80":232,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":234,"_71":-5,"_80":235,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":238,"_71":-5,"_80":239,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":241,"_71":-5,"_80":242,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":245,"_71":-5,"_80":246,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":249,"_71":-5,"_80":250,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":253,"_71":-5,"_80":254,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":257,"_71":-5,"_80":258,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":260,"_71":-5,"_80":261,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":263,"_71":-5,"_80":264,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":266,"_71":-5,"_80":267,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":270,"_71":-5,"_80":271,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":273,"_71":-5,"_80":274,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":276,"_71":-5,"_80":277,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":279,"_71":-5,"_80":280,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":282,"_71":-5,"_80":283,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":285,"_71":-5,"_80":286,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":289,"_71":-5,"_80":290,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":292,"_71":-5,"_80":293,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":295,"_71":-5,"_80":296,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":298,"_71":-5,"_80":299,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":301,"_71":-5,"_80":302,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":305,"_71":-5,"_80":306,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":308,"_71":-5,"_80":309,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":312,"_71":-5,"_80":313,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":315,"_71":-5,"_80":316,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":318,"_71":-5,"_80":319,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":322,"_71":-5,"_80":323,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":326,"_71":-5,"_80":327,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":330,"_71":-5,"_80":331,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":333,"_71":-5,"_80":334,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":336,"_71":-5,"_80":337,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":339,"_71":-5,"_80":340,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":342,"_71":-5,"_80":343,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":345,"_71":-5,"_80":346,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":348,"_71":-5,"_80":349,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":352,"_71":-5,"_80":353,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":355,"_71":-5,"_80":356,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":358,"_71":-5,"_80":359,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":361,"_71":-5,"_80":362,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":364,"_71":-5,"_80":365,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":368,"_71":-5,"_80":369,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":371,"_71":-5,"_80":372,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":374,"_71":-5,"_80":375,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":377,"_71":-5,"_80":378,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":380,"_71":-5,"_80":381,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":384,"_71":-5,"_80":385,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":78,"_44":387,"_71":-5,"_80":388,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":391,"_71":-5,"_80":392,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":394,"_71":-5,"_80":395,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":397,"_71":-5,"_80":398,"_73":82,"_94":-5,"_95":-5,"_85":-5},{"_77":91,"_44":400,"_71":-5,"_80":401,"_73":82,"_94":-5,"_95":-5,"_85":-5},"publicChapterContent","nextChapter","sections",[],"routes/_store+/_layout","routes/_store+/_readers+/_layout","actionData","errors"]

Cover Material, Copyright, and License

Table of Contents

Cover Material, Copyright, and License

Preface

A Request from the Author

Structure of the Book

Code Examples

Functional Programming Requires a Different Mind Set

eBooks Are Living Documents

Setting Up Your Development Environment

Why Haskell?

Enjoy Yourself

Acknowledgements

Section 1 - Tutorial

Tutorial on Pure Haskell Programming

Interactive GHCi Shell

Introduction to Haskell Types

Functions Are Pure

Using Parenthesis or the Special $ Character and Operator Precedence

Lazy Evaluation

Understanding List Comprehensions

Haskell Rules for Indenting Code

Understanding let and where\n

Conditional do Expressions and Anonymous Functions

Simple Pattern Matching

Pattern Matching With Guards

Case Expressions

If Then Else expressions

Maps

Sets

More on Functions

Comments on Dealing With Immutable Data and How to Structure Programs

Error Handling

Testing Haskell Code

Pure Haskell Wrap Up

Tutorial on Impure Haskell Programming

Hello IO () Monad

A Note About >> and >>= Operators

Console IO Example with Stack Configuration

File IO

Error Handling in Impure Code

Network IO

Server Using network-simple Library

Client Using network-simple Library

A Haskell Game Loop that Maintains State Functionally

Efficiency of Haskell Strings

A More Detailed Look at Monads

State Monad

Using Applicative Operators <$> and <*>: Finding Common Words in Files

List Comprehensions Using the do Notation

Dealing With Time

Using Debug.Trace

Wrap Up

Section 2 - Cookbook

Using the OpenAI Large Language Model APIs in Haskell

Example Client Code

Adding a Simple Application: Find Place Names in Input Text

A Web Application For Using the Google Gemini APIs

Web App Wrap Up and a Description of Scotty Web Framework

Command Line Utility To Use the Google Gemini APIs

Command Line Tool to Use Local Ollama LLM Server

Using the Brave Search API

Haskell Code Description for Brave Search Suggestions

Functionality:

Key Features:

Libraries Used:

Language Extensions:

Overall:

Text Processing

CSV Spreadsheet Files

JSON Data

Cleaning Natural Language Text

Natural Language Processing Tools

Resolve Entities in Text to DBPedia URIs

Bag of Words Classification Model

Text Summarization

Part of Speech Tagging

Natural Language Processing Wrap Up

Linked Data and the Semantic Web

The SPARQL Query Language

\n`main` Function