Implementing OPS5 in Racket: A Forward-Chaining Production System

I converted OPS5 from Common Lisp to MIT Scheme in the early 1980s. The code for this chapter is that code, converted to run in Racket. Dear reader, even though I spent years of my life working on rule based symbolic AI, now I don’t really recommend rule based systems for practical work.

Dear reader, while the material in this chapter is important to me for historic reasons, the next chapter uses a modern logic library Racklog to solve the same two example problems covered in this chapter.

OPS5 was written by Charles Forgy (Carnegie Mellon University) and is the classic production-system language. The original Common Lisp code was “very hackable” and I heavily modified it twice for projects at SAIC in the 1980s. I hope that Racket developers find this version converted to Racket to also be “hackable.”

An expert system shell is a program that runs rules over a set of facts. You write the rules. The shell decides which rules apply, picks one, fires it, and repeats.

The Racket conversion does not change the algorithm. The interesting work is the algorithm itself: how a forward-chaining rule engine matches rule conditions against facts without scanning every fact on every cycle. The answer is the Rete algorithm, and it is the heart of this chapter.

Production Systems and Forward Chaining

A production system has three parts:

  1. Working memory: a set of facts. In OPS5 each fact is a tagged list, such as (card heart 10) or (monkey ^ at 5-7 ^ on couch ^ holds nothing).
  2. Productions: rules. Each production has a left-hand side (LHS), a pattern that matches working memory, and a right-hand side (RHS), actions that change working memory.
  3. The inference engine: the loop that matches the LHS of every production against working memory, chooses one matching production, and runs its RHS.

Forward chaining means the engine works from facts to conclusions. It looks at what is true now, finds rules whose conditions hold, fires them, and lets the new facts trigger more rules. It runs until no rule matches or until a rule halts it. This is the opposite of backward chaining, which starts from a goal and asks which rules could prove it.

The engine repeats one cycle:

  1. Match: find all productions whose LHS is satisfied by the current working memory. The set of satisfied instantiations is the conflict set.
  2. Resolve: pick one instantiation from the conflict set.
  3. Act: run that instantiation’s RHS, which adds, removes, or changes facts.
  4. Repeat, unless a halt fires or the conflict set is empty.

The hard part is step 1. A naive engine, on every cycle, tests every production against every combination of facts. If a production has several condition elements, the engine must try combinations of facts across those elements. The cost grows fast. We fix this with the Rete algorithm.

The Rete Algorithm

Rete (Latin for “net”) solves the match problem by storing partial matches in a network and updating them incrementally. When a fact enters or leaves working memory, the engine pushes that one change through the network instead of re-scanning everything.

The naive cost of matching is roughly Code Test per cycle, where R is the number of rules, W is the number of working memory elements, and c is the number of condition elements in a rule. Rete turns this into work proportional to the size of the change, not the size of memory.

The network has two halves.

The alpha network tests one condition element at a time. Each node checks one field of one fact against a constant or a variable binding: “is the suit equal to heart,” “is the number greater than 7.” A fact that passes all alpha tests for a condition element enters that element’s alpha memory as a token.

The beta network joins tokens across condition elements. A beta node takes tokens from two parent memories and combines the ones that agree on their shared variables. The join tests live here: “the <num> bound in condition 1 equals the <num> bound in condition 2.” Beta memories store the joined, partial matches so later joins reuse them.

Three extra node types matter:

  • A memory node (&mem) holds tokens so downstream joins can read them.
  • A join node (&and) combines a new token with the tokens in the opposite memory, running the inter-condition tests.
  • A negative node (&not) handles a negated condition element, one prefixed with -. It fires only when no fact matches the negated pattern.

At the bottom of the network sits a terminal node (&p) for each production. When a full match reaches a &p node, the engine adds it to the conflict set. When a match is withdrawn, the engine removes it.

Rete also shares nodes across productions. If two rules test (card <suit> <num>), they can share the alpha subnetwork for that pattern. The stats the engine prints separate real nodes from virtual nodes: virtual counts every node the compiler builds, real counts the unique ones after sharing. We will see that in this conversion the two counts come out equal, because the node-sharing lookup does not find matches.

Conflict Resolution

Many productions may match at once. OPS5 picks one with a strategy. This code supports two.

  • LEX (lexicographic): among matching instantiations, prefer the one whose matched facts are most recent. If two share the same recency pattern, the engine breaks the tie by sorting the time tags of the matched facts and comparing the sorted lists element by element.
  • MEA (means-ends analysis): like LEX, but it first compares only the time tag of the first condition element. The first condition element acts as the goal condition and dominates the choice.

Both strategies also respect refraction: an instantiation that has already fired cannot fire again on the same facts. This stops a rule from firing forever on unchanged working memory. The engine records fired instantiations and rejects them when they reappear.

The OPS5 Language

An OPS5 program is a sequence of forms. The core forms are literalize, p, make, and run.

literalize declares a class of fact and the attribute names it uses. This lets the compiler assign fixed field positions to attributes so a pattern can name a field by attribute instead of by position.

1 (literalize card
2   suit
3   number)

A production is written with p:

1 (p production-name
2    condition-element
3    condition-element
4    -negated-condition-element
5 -->
6    action
7    action)

A condition element is a list. The first symbol is the class. Attributes follow the ^ marker, with spaces around it:

1 (card ^ suit <suit> ^ number <num>)

Variables are atoms in angle brackets: <suit>, <num>. The first use of a variable binds it. Later uses test equality with the bound value, unless a predicate says otherwise. The predicates are =, <>, <, <=, >, >=, and <=>. For example ^ at <> <p> means “the at attribute is not equal to the value bound to <p>.”

Curly braces { and } mark a condition-element variable, which binds a whole fact so the RHS can refer to it. In this Scheme port the braces must be written as quoted strings, "{" and "}", because the reader would otherwise treat them specially.

The RHS actions are make, modify, ops-remove, ops-write, bind, cbind, halt, and compute. make adds a fact. modify changes one field of an existing fact and is how the engine updates state. ops-write prints, with (crlf) for a newline. halt stops the run.

With the language in hand, we can read the two example programs that ship with the system.

The Example Programs

The directory holds two .ops files. They are the data this system runs on, and they show two different uses of OPS5: one does pattern finding, the other does planning.

draw.ops: finding pairs in a poker hand

This program looks at a hand of cards and finds pairs and three of a kind. The facts are simple: a goal flag and one fact per card. Here is the complete file.

 1 ;; Sample OPS5 program for Draw
 2 
 3 (i-g-v)
 4 
 5 (p look-for-pairs
 6    (goal start)
 7    (card <suit> <num>)
 8   -(three)
 9    (card "{" <suit2> <> <suit> "}" <num>)
10   -(pair <suit2> <suit> <num>)
11 -->
12    (make pair <suit> <suit2> <num>)
13    (ops-write (crlf) found a pair <suit> <num> <suit2>))
14 
15 (p look-for-three-of-a-kind
16    (goal start)
17    (card <suit> <num>)
18   -(four)
19    (card "{" <suit2> <> <suit> "}" <num>)
20    (card <suit3> "{" <> <suit2> "}" )
21   -(three <any-suit1> <any-suit2> <any-suit3> "{" <num10> >= <num> "}" )
22 -->
23    (make three <suit> <suit2> <suit3> <num>)
24    (ops-write three of a kind <suit> <suit2> <suit3> <num>))
25 
26 (make goal start)
27 (make card heart 10)
28 (make card diamond 10)
29 (make card club 10)
30 (make card diamond 4)

Read the first production, look-for-pairs. Its LHS needs five things, in order: a goal fact equal to start; a card fact that binds <suit> and <num>; the absence of any three fact (the - prefix negates the element); a second card whose suit is not equal to <suit> (the <> <suit> test) but whose number equals <num>; and the absence of a pair fact already recording this pair. The negated elements stop the rule from finding the same pair twice or repeating work a three-of-a-kind rule already covered.

When all five conditions hold, the RHS makes a pair fact and prints the find. The new pair fact feeds back into the negated condition, so that combination will not fire again. This feedback, plus refraction, is how the system converges to “no production true” and stops.

The second production, look-for-three-of-a-kind, extends the same idea to three cards. It binds three suits, tests that the second and third differ from the first and from each other, and guards against re-deriving a three that already exists.

The five make calls at the end seed working memory with the goal and the four cards. When you run the program, the rules fire and the hand gets analyzed.

monkey.ops: the monkey and the bananas

The second program is the classic monkey-and-bananas planning problem. A monkey sits on a couch. Bananas hang on the ceiling out of reach. A light ladder lies on the floor elsewhere. The monkey must form a plan: get off the couch, walk to the ladder, pick it up, carry it under the bananas, climb it, and grab the bananas.

The program declares three classes with literalize:

 1 (literalize start )
 2 (literalize monkey
 3     at
 4     on
 5     holds)
 6 
 7 (literalize object
 8     name
 9     at
10     weight
11     on)
12 
13 (literalize goal
14     status
15     type
16     object
17     to)

A monkey fact says where the monkey is (at), what it sits on (on), and what it holds (holds). An object fact names a thing, locates it, gives its weight, and says what it rests on. A goal fact drives the action: it has a status (active or satisfied), a type (holds, move, walk-to, on), the object it concerns, and a destination to.

The rules chain through subgoals. Here is the first production, which reacts to wanting something on the ceiling.

1 (p mb1
2     (goal ^ status active ^ type holds ^ object <w>)
3     (object ^ name <w> ^ at <p> ^ on ceiling)
4     -->
5     (ops-write (crlf)  Since the <w> are on the ceiling at position <p> )
6     (ops-write (crlf)  I would like to move  the ladder under them.)
7     (make goal ^ status active ^ type move ^ object ladder ^ to <p>))

Read the LHS. It needs an active goal of type holds for some object <w>, and an object fact proving <w> is on the ceiling at position <p>. The RHS prints a line and makes a new active goal: move the ladder to <p>. That new goal triggers later rules.

The grabbing rule, mb4, is the payoff production. It fires when the monkey is on the ladder, under the bananas, with empty hands.

1 (p mb4
2     (goal ^ status active ^ type holds ^ object <w>)
3     (object ^ name <w> ^ at <p> ^ on ceiling)
4     (object ^ name ladder ^ at <p>)
5     (monkey ^ on ladder ^ holds nil)
6     -->
7     (ops-write (crlf) I have the <w>  in hand)
8         (modify 4 ^ holds <w>)
9         (modify 1 ^ status satified))

The (modify 4 ^ holds <w>) action changes the fourth matched fact, the monkey fact, so the monkey now holds the bananas. The (modify 1 ^ status satified) action marks the goal satisfied. (The original source spells this satified; the code is unchanged from the historical program.)

The movement rules use the brace condition-element variable and inequality tests. This rule walks the monkey to a destination when it is on the floor, not already there, and holding nothing.

1 (p mb12
2     (goal ^ status active ^ type walk-to ^ object <p>)
3     (monkey ^ on floor ^ at "{"  <c> <> <p>  "}"  ^ holds nothing)
4     -->
5     (ops-write (crlf) I will walk over to <p>)
6         (modify 2 ^ at <p>)
7         (modify 1 ^ status satisfied))

The "{" <c> <> <p> "}" part binds the monkey’s current position to the whole-fact variable <c> while testing that it is not equal to the goal location <p>. The RHS moves the monkey and satisfies the walk goal.

A carrying rule, mb13, is the partner: if the monkey is holding something when it walks, the carried object moves with it. There are rules for jumping down to the floor (mb14), for needing free hands to climb (mb16, mb17), and for dropping a held object to free the hands (mb18). Each rule makes or satisfies a goal, and the goals drive the next rule.

The program ends with a starter production, t1, that creates the initial world when it sees a start fact.

 1 (p t1
 2     (start 1)
 3     -->
 4     (make monkey ^ at 5-7 ^ on couch ^ holds nothing)
 5     (ops-write (crlf) I am a monkey lying on the couch)
 6     (make object ^ name couch ^ at 5-7 ^ weight heavy)
 7     (ops-write (crlf)  "... a heavy couch")
 8     (make object ^ name bananas ^ on ceiling ^ at 2-2)
 9     (ops-write (crlf) there are some bananas on the ceiling at position 2-2)
10     (make object ^ name ladder ^ on floor ^ at 9-5 ^ weight light)
11     (ops-write (crlf) there is a ladder on the floor at position 9-5)
12 
13     (make goal ^ status active ^ type holds ^ object bananas)
14     (ops-write (crlf) I sure would like those bananas )
15     (ops-write (crlf) (crlf) "The action begins:" (crlf)))

The monkey starts at 5-7 on the couch. The bananas are on the ceiling at 2-2. The ladder is on the floor at 9-5. The first active goal is to hold the bananas. You trigger t1 by making a start fact, and the plan unfolds from there.

The full monkey.ops file has 18 productions (mb1 through mb18) plus the starter. Each follows the same shape as the three shown here: match a goal and some world facts, print a line, and make or satisfy a subgoal.

The Racket Conversion: One File of Pure Code

Dear reader, I had many problems with the Racket conversion. For a long time I worked around them with a hack: the whole system lived inside one giant string, which the driver wrote to a temporary file and loaded into a dedicated namespace. That preserved the sequential, load-time semantics the code relies on, but it was ugly and hard to edit. The fix turned out to be one line: the file now starts with #lang racket/load and contains nothing but plain Racket code.

The original OPS5-in-Scheme code was split across six files: a compatibility layer, the top-level commands, the LHS compiler, the Rete network, the RHS actions, and the literalize support. A driver loaded them in order into a namespace. The Racket version, ops5.rkt, keeps that structure as six clearly marked sections in one file.

Why racket/load instead of plain #lang racket? A Racket module rejects a second definition of an identifier at the top level (“identifier already defined”), and the OPS5 source redefines names freely — both its own helper names across sections and names like append and member that it deliberately replaces with lenient versions. The racket/load language gives the file load-like top-level semantics: each form is evaluated in order as if loaded, redefinition is allowed, and eval at run time sees the definitions made so far. That is exactly the behavior the old namespace hack was simulating.

1 #lang racket/load
2 ;; ops5.rkt -- the complete OPS5-in-Racket system as pure code.
3 ;; racket/load gives load-like top-level semantics: redefinitions and
4 ;; (require ...) forms behave as they did under load.rkt.

There is one subtlety worth knowing about, because it bit me during the cleanup. Under racket/load, free references in a definition are bound when that form is expanded, in file order. The OPS5 function remove-duplicates is defined near the end of the file, but old-literalize, defined earlier, calls it — and since remove-duplicates also exists in racket/base, the call site captured Racket’s strict version, which rejects the mutable pairs this code uses. The fix was to rename the OPS5 version to ops5-remove-duplicates, a unique name, so the call becomes a forward reference resolved at run time.

1 (define (ops5-remove-duplicates lst)
2   ;; the atom base case must return '() (MIT nil doubles as empty list
3   ;; and false); returning #f would make appended lists improper
4   (cond ((null? lst) '())
5         ((atom? lst) #f)
6         ((member (car lst) (cdr lst)) (ops5-remove-duplicates (cdr lst)))
7         (t (cons (car lst) (ops5-remove-duplicates (cdr lst))))))

The compatibility layer

The first section of the file is the compatibility layer (the old compat.rkt). It bridges MIT Scheme and Racket. The original code assumes mutable pairs, a t and nil that differ from Racket’s #t and '(), and a set of list functions with lenient semantics. Racket’s pairs are immutable, so the layer imports mutable pairs from the r5rs language.

 1 (require (except-in r5rs eval lambda)
 2                   )
 3 (require (only-in r5rs
 4                   cons car cdr set-car! set-cdr! pair? null? list list? reverse
 5                   caar cadr cdar cddr
 6                   ;; ... about 30 more imported bindings elided ...
 7                   ))
 8 
 9 (define t #t)
10 (define nil '())
11 
12 (define (atom? x) (not (pair? x)))
13 (define proper-list? list?)
14 (define listp list?)
15 (define symbolp symbol?)

The list functions need lenient versions because the OPS5 code passes #f and atoms where Racket’s strict versions would raise contract errors. Each lenient function is defined under an implementation name and then aliased, so the recursive calls inside the body bind to the lenient version, not to a previously imported strict one.

 1 (define (mapcar-impl f l)
 2   (cond ((null? l) '())
 3         ((pair? l) (cons (f (car l)) (mapcar-impl f (cdr l))))
 4         (else #f)))
 5 (define mapcar mapcar-impl)
 6 
 7 (define (member-impl x l)
 8   (cond ((pair? l) (if (equal? x (car l)) l (member-impl x (cdr l))))
 9         (else #f)))
10 (define member member-impl)

A while macro handles a subtle truthiness gap. MIT Scheme treats the empty list as false. Racket treats the empty list as true. So while must stop not only on #f but also on '().

1 (define (mit-true? x) (and x (not (null? x))))
2 (define-syntax-rule (while test body ...)
3   (let loop () (when (mit-true? test) body ... (loop))))

This gap, between '() as false and '() as true, is the single most common source of bugs when porting old Scheme to Racket. The compatibility layer localizes the fix. You will see the same care in the compiler and network code, where comments note each place the original relied on MIT’s empty-list-is-false rule.

The Top-Level Commands as Macros

The user-facing OPS5 commands are p, make, modify, run, wm, and strategy. In the original code these were MIT macro forms. In Racket they become define-syntax transformers that build their expansions as data and quote the arguments. This preserves the key behavior: a production form reaches the compiler unevaluated, so the compiler sees the literal pattern.

1 (define-syntax p
2   (lambda (stx)
3     (syntax-case stx ()
4       [(_ . rest)
5        (datum->syntax stx
6          (list 'old-p (list 'quote (syntax->datum stx))))])))

The p macro rewrites (p name lhs --> rhs) into (old-p '(p name lhs --> rhs)). The whole production, unevaluated, goes to old-p. The other macros follow the same pattern. make quotes each argument and hands them to old-make.

 1 (define-syntax make
 2   (lambda (stx)
 3     (syntax-case stx ()
 4       [(_ . args)
 5        (datum->syntax stx
 6          (cons 'old-make
 7                (map (lambda (a) (list 'quote a))
 8                     (syntax->datum #'args))))])))
 9 
10 (define (old-make . l)
11   (!reset)
12   (eval-args l)
13   (!assert))

old-make is the runtime half. It resets the result array that builds a new fact, evaluates the arguments into that array, and asserts the assembled fact into working memory. We will see !reset, eval-args, and !assert when we reach the RHS.

Compiling a Production into the Network

The LHS compiler turns a production’s pattern into Rete nodes. The entry point is old-p, which prints the production name, finishes any pending literalize declarations, and calls compile-production.

1 (define (old-p z)
2   (write (car z)) (newline)
3   (set! z (cdr z))
4   (finish-literalize)
5   (write '*)
6   (let ((flag nil) (temp nil))
7     (set! temp (compile-production (car z) (cdr z)))
8     (set! flag t)
9     (display "compiled") (display  (car z))))

compile-production records the production name and calls cmp-p, the real compiler.

 1 (define (cmp-p name matrix)
 2   (let ((m nil) (bakptrs nil))
 3         (cond ((or (null? name) (proper-list? name))
 4            (%error "Illegal production name" name)))
 5         (prepare-lex matrix)
 6         (excise-p name)
 7         (set! bakptrs nil)
 8         (set! *pcount* (+ 1 *pcount*))
 9         (set! *feature-count* 0)
10         (set! *ce-count* 0)
11         (set! *vars* nil)
12         (set! *ce-vars* nil)
13         (set! *rhs-bound-vars* nil)
14         (set! *rhs-bound-ce-vars* nil)
15         (set! *last-branch* nil)
16         (set! m *matrix*)
17         (while (not (equal? '--> (peek-lex)))
18          (begin
19           (and (atom? *matrix*) (%error "No '-->' in production" m))
20           (cmp-prin)
21           (set! bakptrs (cons *last-branch* bakptrs))))
22         (lex)
23         (check-rhs *matrix*)
24         (link-new-node (list '&p
25                              *feature-count*
26                              name
27                              (encode-dope)
28                              (encode-ce-dope)
29                              (eval (cons 'lambda (cons nil *matrix*)))))
30         (putprop name (cdr (reverse bakptrs)) 'backpointers)
31         (putprop name *last-node* 'topnode)))

Read the loop. It reads tokens until it hits -->. For each condition element it calls cmp-prin, which builds alpha test nodes for that element and, for every element after the first, a beta node that joins it to the previous ones. Each *last-branch* is the first node of one condition element’s subnetwork; the list of these becomes the production’s backpointers, used later by the (matches) debug command.

After -->, the compiler checks the RHS, then links a terminal &p node. That node carries the production name, the variable dope (which field each variable came from), the condition-element dope, and the RHS itself wrapped in a lambda. The lambda is the code the engine runs when this production fires.

The single-condition-element compiler, cmp-ce, reads the element and walks its fields. For each field it dispatches to a node builder based on what the field is.

 1 (define (cmp-element)
 2         (and (equal? (car *curcond*) '^) (cmp-tab))
 3         (cond ((equal? (car *curcond*) leftcurly) (cmp-product))
 4               (t (cmp-atomic-or-any))))
 5 
 6 (define (cmp-atomic-or-any)
 7         (cond ((equal? (car *curcond*) '<<) (cmp-any))
 8               (t (cmp-atomic))))
 9 
10 (define (cmp-atomic)
11   (let ((test nil) (x (car *curcond*)))
12         (cond ((eq? x '=)   (set! test 'eq) (sublex))
13               ((eq? x '<>)  (set! test 'ne) (sublex))
14               ((eq? x '<)   (set! test 'lt) (sublex))
15               ((eq? x '<=)  (set! test 'le) (sublex))
16               ((eq? x '>)   (set! test 'gt) (sublex))
17               ((eq? x '>=)  (set! test 'ge) (sublex))
18               ((eq? x '<=>) (set! test 'xx) (sublex))
19               (t (set! test 'eq)))
20         (cmp-symbol test)))

A field with a predicate like <> sets the test type and reads on. Then cmp-symbol decides whether the field is a variable, a number, or a constant, and links the right alpha node. Constants and numbers become one-argument test nodes; variables become either a binding (first use) or a two-argument test node that compares two fields (later use).

The node names are built by concat, which packs the test type, the comparison kind, and the operand type into one symbol. teqa means “test equal atom,” tnea means “test not-equal atom,” tnen means “test not-equal number,” and so on. The first letter group is the test, the middle is the operator, the last is the operand kind: a for atom, n for number, s for a field-to-field (same) test, b for a field-to-field beta test.

Node sharing happens in the linker. Before creating a node, the compiler checks the parent’s existing children for an equivalent one.

1 (define (link-left pred succ)
2   (let ((a (left-outs pred)) (r nil))
3         (set! r (find-equiv-node succ a))
4         (if r
5             r
6             (begin
7                 (set! *real-cnt* (add1 *real-cnt*))
8                 (attach-left pred succ)
9                 succ))))

If an equivalent node already exists, the compiler reuses it and does not increment the real-node count. This is how two rules that share (card <suit> <num>) end up sharing the alpha nodes for that pattern.

The Network Interpreter

When a fact enters or leaves working memory, the engine calls match to push the change into the network from the top.

1 (define (match flag wme)
2   (sendto flag (list wme) 'left (list *first-node*)))

*first-node* is a &bus node, the root that fans every fact out to all top-level alpha subnetworks. The &bus node unpacks the fact into the global field registers *c1*, *c2*, and so on, then evaluates its child nodes. Alpha test nodes read those registers.

A constant-equality test node is short. teqa reads its register, compares to the constant, and, if equal, forwards the fact to its outputs.

1 (define (teqa outs register constant)
2   (and (equal? (local-eval register) constant) (eval-nodelist outs)))

local-eval looks up a register symbol like *c2* in the current namespace to get the field value. When the test passes, eval-nodelist runs the child nodes, which are themselves functions stored as data in the node list.

A memory node stores the tokens that pass through it, so joins can read them later.

 1 (define (&mem left-outs right-outs memory-list)
 2   (let ((fp #f) (dp #f))
 3        (cond (*sendtocall*
 4               (set! fp *flag-part*)
 5               (set! dp *data-part*))
 6              (t
 7               (set! fp *alpha-flag-part*)
 8               (set! dp *alpha-data-part*)))
 9        (sendto fp dp 'left left-outs)
10        (add-token memory-list fp dp #f)
11        (sendto fp dp 'right right-outs)))

The flag part says whether this is an add (new) or a remove (#f or old). The data part is the token, the list of facts matched so far. add-token updates the memory list under that flag: new inserts, #f removes, old is a no-op marker. The node then forwards the token to both its left and right outputs.

The join node, &and, does the real beta work. It takes a new token on one side and scans the opposite memory for tokens that pass the inter-condition tests.

1 (define (&and outs lpred rpred tests)
2   (let ((mem #f))
3        (cond ((eq? *side* 'right) (set! mem (memory-part lpred)))
4              (t (set! mem (memory-part rpred))))
5        (cond ((not mem) #f)
6              ((eq? *side* 'right) (and-right outs mem tests))
7              (t (and-left outs mem tests)))))

If the new token arrived on the right side, the join scans the left parent’s memory, and vice versa. For each stored token, it runs the tests. A test is a beta predicate like teqb applied to two fields pulled from the two tokens by gelm, which decodes a packed (condition-element, field) index. When all tests pass, the join concatenates the two tokens into a longer one and forwards it downstream. That longer token is a fuller partial match. It flows to the next join, or to the terminal node.

The terminal &p node is where a complete match becomes a conflict-set entry.

 1 (define (&p rating name var-dope ce-var-dope rhs)
 2   (let ((fp #f) (dp #f))
 3         (cond (*sendtocall*
 4                (set! fp *flag-part*)
 5                (set! dp *data-part*))
 6               (t
 7                (set! fp *alpha-flag-part*)
 8                (set! dp *alpha-data-part*)))
 9         (and (memq fp '(#f old)) (removecs name dp))
10         (and fp (insertcs name dp rating))))

On an add, insertcs adds the instantiation to the conflict set. On a remove, removecs takes it out. The instantiation is the production name plus the data part, the list of matched facts that will bind the RHS variables.

Working Memory

Working memory is a hash of fact lists. The hash key is the first symbol in the fact, found by wm-hash, so facts cluster by class. Each fact carries a time tag, a number that records when it was added. Time tags drive LEX and MEA conflict resolution.

Adding a fact runs the network with the new flag and records the change for undo.

 1 (define (add-to-wm wme override)
 2   (let ((fa #f) (z #f) (part #f) (timetag #f) (port #f))
 3     (set! *critical* t)
 4     (set! *current-wm* (1+ *current-wm*))
 5     (and (> *current-wm* *max-wm*) (set! *max-wm* *current-wm*))
 6     (set! *action-count* (1+ *action-count*))
 7     (set! fa (wm-hash wme))
 8     (or (memq fa *wmpart-list*)
 9         (set! *wmpart-list* (cons fa *wmpart-list*)))
10     (set! part (get fa 'wmpart*))
11     (cond ((and override (not (null? override)))
12              (set! timetag override))
13           (t (set! timetag *action-count*)))
14     (set! z (cons wme timetag))
15     (putprop fa (cons z part) 'wmpart*)
16     (record-change '=>wm *action-count* wme)
17     (match 'new wme)
18     (set! *critical* #f)
19     (cond ((and *in-rhs* *wtrace*)
20            (newline)
21            (write "Adding to WM: ")
22            (write wme)
23            (newline)))))

The fact gets a time tag (the current action count, unless an override is given, as in refresh). It is stored under its class key. Then (match 'new wme) pushes it into the Rete network, so every production that now matches it gets a conflict-set entry. Removing a fact does the reverse: it calls (match #f wme) to withdraw matches, then deletes the fact from its class list.

The (wm) command prints working memory by mapping over the class buckets.

 1 (define (old-wm a)
 2   (mapc (lambda (z) (ppelm z))
 3         (get-wm a)))
 4 
 5 (define (get-wm z)
 6   (set! *wm-filter* z)
 7   (set! *wm* #f)
 8   (mapwm get-wm2)
 9   (let ((temp *wm*))
10     (set! *wm* #f)
11     temp))

Each fact prints as (time-tag (fact)). The optional argument filters by time tag. With no argument, wm prints every fact.

The Recognize-Act Loop

The run command sets the cycle budget and calls do-continue, which processes pending changes and calls main.

 1 (define (old-run z)
 2   (set! *remaining-cycles* z)
 3   (do-continue #f))
 4 
 5 (define (do-continue wmi)
 6     (cond (*critical*
 7            (newline)
 8            (write "Warning: network may be inconsistent")))
 9     (process-changes wmi #f)
10     (print-times (main)))

main is the cycle. It picks an instantiation, fires it, and loops.

 1 (define (main)
 2   (let ((instance #f) (r #f))
 3 
 4      (define (loop)
 5         (set! *phase* 'conflict-resolution)
 6            (cond ((and #f (equal? (peek-char) 13))  ;; skip this logic because of #F clause
 7                (set! *halt-flag* t)
 8                (set!  *break-flag* t)
 9                (read-char)
10                (newline)
11                (display "Interrupted by a keystroke")
12                (newline))
13               (t
14                 (cond (*halt-flag*
15                        (set! r "End -- explicit halt")
16                        (finis))
17                       ((zero? *remaining-cycles*)
18                        (set! r "***break***")
19                        (set! *break-flag* t)
20                        (finis))
21                       (*break-flag*
22                        (set! r "***break***")
23                        (finis))
24                       (t
25                         (set! *remaining-cycles* (-1+ *remaining-cycles*))
26                         (set! instance (conflict-resolution))
27                         (cond ((not instance)
28                                (set! r "End -- no production true")
29                                (finis))
30                               (t
31                                 (set! *phase* (car instance))
32                                 (accum-stats)
33                                 (eval-rhs (car instance) (cdr instance))
34                                 (check-limits)
35                                 (and
36                                  (broken (car instance))
37                                  (set! *break-flag* t))
38                                 (loop))))))))
39   (define (finis)
40      (set! *p-name* #f)
41      r)
42 
43   (set! *halt-flag* #f)
44   (set! *break-flag* #f)
45   (set! instance #f)
46   (loop)))

Each iteration calls conflict-resolution to pick the best instantiation. If there is none, the loop ends with “End – no production true.” If a halt flag is set, it ends with “End – explicit halt.” Otherwise it runs the RHS and loops.

Conflict resolution is a tournament. best-of walks the conflict set and keeps the winner under the strategy’s comparison.

 1 (define (conflict-resolution)
 2   (let ((best #f) (len (length *conflict-set*)) (temp #f))
 3     (cond ((> len *max-cs*) (set! *max-cs* len)))
 4     (set! *total-cs* (+ *total-cs* len))
 5     (cond ((pair? *conflict-set*)
 6            (set! best (best-of *conflict-set*))
 7            (set! *conflict-set* (delq best *conflict-set*))
 8            (set! temp (pname-instantiation best)))
 9           (t temp #f))
10     temp))

Each entry in the conflict set is ((p-name . data) (sorted time tags) rating). The order-tags function builds the sorted time-tag list, and it differs by strategy.

1 (define (order-tags dat)
2   (let ((tags #f))
3     (while (and (not (atom? dat)) (not (null? dat)))
4        (begin
5            (set! tags (cons (creation-time (safe-car dat)) tags))
6            (set! dat (cdr dat))))
7     (cond ((eq? *strategy* 'mea)
8            (cons (safe-car tags) (dsort (safe-cdr tags))))
9           (t (dsort tags)))))

Under LEX, all time tags are sorted and compared as a list. Under MEA, the first condition element’s time tag is pulled out and compared first, and only the rest are sorted for tie-breaking. That first tag is the goal condition’s tag, so MEA favors the instantiation whose goal fact is newest. The chosen entry is removed from the conflict set so it will not be picked again this pass; refraction will keep it out if the facts have not changed.

Running the RHS

When an instantiation fires, eval-rhs binds the matched facts to their variables and runs the production’s RHS lambda.

 1 (define (eval-rhs pname data)
 2   (let ((node nil) (port nil) (eval-expression nil))
 3     (cond (*ptrace*
 4             (newline) (display *cycle-count*) (display ". ")
 5             (display pname) (time-tag-print data)))
 6     (set! *data-matched* data)
 7     (set! *p-name* pname)
 8     (set! *last* nil)
 9     (set! node (get pname 'topnode))
10     (init-var-mem (cadddr node))
11     (init-ce-var-mem (cadr (cdddr node)))
12     (begin-record pname data)
13     (set! *in-rhs* t)
14     (set! eval-expression (caddr (cdddr node)))
15     (eval-expression)
16     (set! *in-rhs* nil)
17     (end-record)))

init-var-mem reads the variable dope and builds an association list mapping each variable to the field value pulled from the matched data. The RHS lambda then runs. Inside it, !varbind looks up variables in that association list. *in-rhs* is set true so that make, modify, and ops-write know they are running inside a firing, not at the top level.

A make action assembles a new fact in a result array and asserts it. !value fills the next slot; !assert turns the array into a list and adds it to working memory.

 1 (define (!value v)
 2   (cond ((> *next-index* *size-result-array*)
 3          (%warn "Index too large" *next-index*))
 4         (t
 5          (and (> *next-index* *max-index*)
 6               (set! *max-index* *next-index*))
 7          (putvector *result-array* *next-index* v)
 8          (set! *next-index* (add1 *next-index*)))))
 9 
10 (define (!assert)
11   (set! *last* (use-result-array))
12   (add-to-wm *last* nil))

A modify action reads the bound condition-element fact, removes it from working memory, copies its fields into the result array, applies the changes, and asserts the result. This remove-then-add is what makes modify propagate through the network: the removal withdraws old matches, and the add creates new ones. That propagation is why firing one rule can trigger the next.

Running the Code

You need Racket. No packages are required; the code uses only the standard library and the r5rs language that ships with Racket.

Run the system from the example directory. Give it a .ops file to load before the REPL starts.

1 racket ops5.rkt draw.ops

The system prints its banner and the compiled production names, then drops you at the OPS5> prompt. Type (run) to fire the productions.

draw.ops

 1 $ racket ops5.rkt draw.ops
 2 ******* Beta test of OPS5 *******
 3 Note: the Scheme version of OPS5 requires curly brakets { and }
 4 to have surrounding double quotes.  Place spaces around the ^tab character.
 5 Copyright 1986, Mark Watson
 6 p
 7 *compiledlook-for-pairsp
 8 *compiledlook-for-three-of-a-kind
 9 OPS5 Scheme interpreter (Racket conversion)
10 Type OPS5 expressions, e.g.:
11   (load "draw.ops")   load a program file
12   (i-g-v)              initialize (or reset) OPS5
13   (p name lhs --> rhs) define a production
14   (make class ...)     add a working-memory element
15   (run)                run the productions
16   (wm)                 print working memory
17   (exit)               leave the REPL
18 
19 OPS5> (run)
20  three of a kind heart diamond diamond 10 three of a kind diamond heart diamond 10
21 three of a kind diamond club diamond 10 three of a kind heart club diamond 10
22 three of a kind club diamond diamond 10 three of a kind club heart diamond 10
23 found a pair club 10 heart
24 found a pair club 10 diamond
25 found a pair diamond 10 club
26 ...
27 found a pair diamond 10 heart
28 End -- no production true
29 
30 (2 productions (42 // 42 nodes))(28 firings (33 RHS actions))
31 (18 Mean working memory size (33 maximum))
32 (16 mean conflict set size (30 maximum))
33 (98 mean token memory size (123 maximum))OPS5>

Before running, you can inspect the facts with (wm).

1 OPS5> (wm)
2 (1 (goal start))
3 (2 (card heart 10))
4 (3 (card diamond 10))
5 (4 (card club 10))
6 (5 (card diamond 4))

The four cards are three tens and a four. The two rules find the pairs among the tens and announce three of a kind in several suit orderings. The output is noisy because the three-of-a-kind rule fires for each ordering of the three suits, and the pair rule fires for each ordering of each pair. The negated guards and refraction eventually stop every rule, and the engine ends with “End – no production true.”

monkey.ops

Start the monkey program and trigger the starter production with (make start 1).

 1 $ racket ops5.rkt monkey.ops
 2 ... banner and 19 compiled production names ...
 3 
 4 OPS5> (make start 1)
 5 OPS5> (run)
 6 
 7 I am a monkey lying on the couch
 8 ... a heavy couch
 9 there are some bananas on the ceiling at position 2-2
10 there is a ladder on the floor at position 9-5
11 I sure would like those bananas
12 
13 The action begins:
14 
15 Since the bananas are on the ceiling at position 2-2
16 I would like to move the ladder under them.
17 since the ladder is light I can move it
18 I think I will walk over to 9-5 to get the ladder
19 since I need to be on the floor to walk
20 I better get to the floor
21 I will jump onto the floor
22 I will walk over to 9-5
23 I picked the ladder off the floor
24 since I can move the ladder to 2-2 I will
25 I will carry ladder to 2-2
26 With the ladder at 2-2
27 I climb onto the ladder to get the bananas .
28 I will need free hands to climb the ladder
29 and it is where I want it
30 since I need my hands free I will put ladder down
31 I will now climb onto ladder
32 what I want to do now is get the bananas
33 End -- no production true
34 
35 (19 productions (212 // 212 nodes))(16 firings (42 RHS actions))
36 (10 Mean working memory size (14 maximum))
37 (2 mean conflict set size (3 maximum))
38 (48 mean token memory size (60 maximum))OPS5>

Run each example in a fresh session. The README warns about two quirks. (i-g-v) excises all loaded productions, so after a reset you must reload the .ops file before (run). And loading a second .ops file without resetting keeps the old working memory, so rules can fire against stale facts. The simple habit is to (exit) and relaunch with the next file.

Interpreting the Results

The monkey trace is a plan. Read it top to bottom. The monkey wants the bananas. Because they are on the ceiling, it decides to move the ladder under them. Because the ladder is light, it can move it, but first it must pick it up, so it decides to walk to the ladder. To walk it must be on the floor, so it jumps off the couch. It walks to the ladder, picks it up, carries it to 2-2, and climbs it. The last firing, mb3, sets a sub-goal to free the monkey’s hands before the grab. No later rule satisfies that sub-goal in this setup, so the engine stops one step short of the grab and ends with no production true. Each line is one production firing, and the chain of goals drives the next line. This is forward chaining producing goal-directed behavior: there is no search tree and no backtracking. The goal facts and the rule order do the work, and when no rule matches the current goal, the run stops.

The final line of each run is a statistics report. It has four parts.

1 (19 productions (212 // 212 nodes))(16 firings (42 RHS actions))
2 (10 Mean working memory size (14 maximum))
3 (2 mean conflict set size (3 maximum))
4 (48 mean token memory size (60 maximum))
  • Productions and nodes: 19 productions fired here. 212 // 212 nodes gives virtual nodes and real nodes. Virtual is every node the compiler built. Real is the unique nodes after sharing. They are equal in this program because no two productions share enough to merge nodes. In a larger program with shared patterns, real would be smaller than virtual.
  • Firings and RHS actions: 16 firings means 16 productions fired. 42 RHS actions counts the individual make, modify, ops-write, and other actions those firings ran. One firing can run several actions, so the action count is higher than the firing count.
  • Working memory size: 10 Mean working memory size (14 maximum). On average 10 facts were in memory across cycles, and at peak there were 14. The monkey program keeps a small world, so these numbers stay low.
  • Conflict set size: 2 mean conflict set size (3 maximum). On average 2 instantiations competed each cycle, peaking at 3. A small conflict set means the strategy had little to choose from, which fits a planning program that mostly has one sensible next step.
  • Token memory size: 48 mean token memory size (60 maximum). Tokens are the partial matches stored in beta memories. This is the memory cost of Rete. It is larger than the working memory size because one fact can participate in many partial matches.

Compare the two programs. The draw program has a large, noisy conflict set (mean 16, max 30) because many suit orderings match at once. The monkey program has a small one (mean 2, max 3) because the goal chain keeps the choices tight. The token memory tells the same story: draw averages 98 tokens, monkey averages 48, even though monkey has far more productions. More rules do not mean more memory when the working memory stays small and the goals channel the matching.

Wrap Up

This chapter walked through a working OPS5 implementation in Racket. The core ideas are independent of the host language.

A production system keeps facts in working memory and runs rules in a match-resolve-act loop. Forward chaining fires rules whose conditions hold and lets the new facts trigger more rules. The cost of matching is the hard problem, and the Rete algorithm solves it by compiling each rule’s pattern into a network of alpha and beta nodes that store partial matches and update incrementally. Conflict resolution picks one instantiation per cycle, using recency (LEX) or a goal-first variant (MEA), and refraction stops repeats. The RHS assembles new facts in a result array and asserts them, and modify does a remove-then-add that pushes changes back through the network.

The Racket conversion did not change any of this. It ported the MIT Scheme dialect to Racket with a compatibility layer that supplies mutable pairs, lenient list functions, and the empty-list-is-false truthiness the original code assumes. It keeps the six source sections in one plain Racket file, and the #lang racket/load language preserves the sequential loading, redefinition, and runtime eval the code relies on. The user-facing commands became Racket macros that pass forms to the compiler unevaluated.

The two examples show the range. draw.ops uses OPS5 for pattern finding over a small fixed set of facts, and the noisy output shows what happens when many orderings match. monkey.ops uses OPS5 for planning, where a chain of goal facts drives a sequence of actions from a couch toward a bunch of bananas, stopping one step short of the grab. The statistics line in each run ties the behavior back to the algorithm: node counts reflect sharing, the conflict set reflects how much the strategy had to choose from, and token memory reflects the cost Rete pays to keep matching cheap.

Optional Practice Problems

These exercises build on the example code in this directory. Run each in a fresh session with racket ops5.rkt your-file.ops.

  1. Four of a kind. The draw.ops program finds pairs and three of a kind but stops there. Add a production look-for-four-of-a-kind that fires when four cards share a number. Seed working memory with four cards of the same number and run it. Use the existing look-for-three-of-a-kind production as a template, and add a -(four) guard to the three-of-a-kind rule so the new rule takes over first.

  2. Detect a flush. Write a production that fires when all cards in the hand share one suit. Add a (make card heart 9) to the seed data so the hand has four hearts, and confirm the flush rule fires. Think about how to test “every card has suit <s>” with the condition elements and variables OPS5 gives you.

  3. Fix the grab. In monkey.ops, the run stops at mb3 with the monkey on the ladder under the bananas, but mb4 never fires, so the monkey never grabs them. Read mb3 and mb4: mb3 makes a goal of type holds with object nil, while mb4 needs object <w> bound to a real object. Explain why mb4 does not match after mb3. Then change mb3 or add a production so the monkey grabs the bananas, and add a (halt) action so the run ends with “End – explicit halt.”

  4. Change the strategy. The default strategy is LEX. Add (strategy mea) near the top of monkey.ops after (i-g-v), run the monkey program, and compare the firing order and the trace to the LEX run. Report which lines change order and why the first condition element’s time tag matters.

  5. A heavy ladder. The ladder is weight light, which is why mb8 can move it. Change the seed in t1 so the ladder is weight heavy. Predict what happens, run it, and explain why the plan stalls. Then add a new object and production that lets the monkey move a heavy object with a tool of your invention.

  6. Blocks world. Write a new .ops file for a simple blocks world. Declare a block class with name, on, and clear attributes. Seed three blocks stacked on a table. Write productions that move the top clear block onto the table or onto another clear block, driven by a goal fact. Run it and confirm the stack unstacks.

  7. Thermostat. Write a .ops file that models a thermostat. Declare a temperature class and a setting class. Seed a current temperature and a desired setting. Write one production that makes a turn-on-heat fact when the temperature is below the setting, and one that makes a turn-off-heat fact when it is at or above. Run it and inspect working memory with (wm).

  8. Read the stats. Add several duplicate conditions across two productions in draw.ops so that node sharing kicks in. Run it and read the virtual // real node counts from the statistics line. Confirm that real is smaller than virtual, and explain which nodes were shared.

  9. Trace a firing. Turn on production tracing with (watch 1) before (run) in the monkey program. The engine prints each firing with its cycle number and time tags. Pick one firing from the trace, identify which production fired and which facts matched, and explain why the conflict resolver chose that instantiation over any competitor in the same cycle.

  10. Negation. The draw.ops pair rule uses -(pair ...) to avoid re-deriving a pair. Remove that negated condition, run the program, and observe what happens to the firing count and the conflict set size. Explain how refraction alone does or does not prevent the explosion.