Inductive Logic Programming with Popper
Machine learning is dominated by connectionist methods like deep artificial neural networks. These models learn by updating millions or billions of continuous numerical weights. While neural networks excel at perception tasks (such as computer vision and speech recognition), they suffer from several drawbacks: they require vast amounts of training data, their internal reasoning is opaque (a “black box”), and they cannot easily incorporate existing domain knowledge.
The material for this chapter is derived and inspired by the Popper inductive logic programming system GitHub repository https://github.com/logic-and-learning-lab/Popper/.
Inductive Logic Programming (ILP) is an alternative approach to machine learning situated at the intersection of machine learning and logic programming. Rather than fitting mathematical functions to numbers, an ILP system learns symbolic rules (Prolog programs) directly from:
- Background Knowledge (BK): Existing facts and rules representing known domain concepts.
- Positive Examples: Examples of the target relationship that the learned program must entail.
- Negative Examples: Examples of the target relationship that the learned program must not entail.
The output is a clean, readable Prolog program that fits the training data perfectly. Because the model is represented as logic rules, it is fully explainable, mathematically exact, and can be immediately loaded into a standard Prolog interpreter for verification or execution.
How Popper Works
Popper is a state-of-the-art ILP engine. It uses a multi-engine loop:
- Generation (ASP): Popper uses Answer Set Programming (ASP) via the Clingo solver to search the space of possible logic programs and generate candidate hypotheses.
- Testing (Prolog): It uses SWI-Prolog to test if the candidate program entails all positive examples and no negative examples.
- Feedback (Learning): If a candidate program fails (e.g., it is too general and entails a negative example, or too specific and misses a positive one), Popper extracts a symbolic constraint explaining why it failed. This constraint is fed back to the ASP solver to prune large portions of the search space, repeating until a minimal, correct solution is found.
The Grandparent Problem Setup
To demonstrate ILP, we want the system to learn the rule defining a grandparent/2 relationship. We provide background knowledge of family structures, lists of positive grandparent pairings, lists of incorrect pairings, and bias parameters.
1. Background Knowledge
We define basic parental relations and genders.
Here is the code in source-code/inductive_logic_popper/bk.pl:
1 parent(pam, bob).
2 parent(tom, bob).
3 parent(tom, liz).
4 parent(bob, ann).
5 parent(bob, pat).
6 parent(pat, jim).
7
8 female(pam).
9 female(liz).
10 female(pat).
11 female(ann).
12 male(tom).
13 male(bob).
2. Positive & Negative Examples
We supply examples of who is and is not a grandparent.
Here is the code in source-code/inductive_logic_popper/exs.pl:
1 pos(grandparent(pam, ann)).
2 pos(grandparent(pam, pat)).
3 pos(grandparent(tom, ann)).
4 pos(grandparent(tom, pat)).
5 pos(grandparent(bob, jim)).
6
7 neg(grandparent(pam, bob)).
8 neg(grandparent(tom, liz)).
9 neg(grandparent(bob, pat)).
10 neg(grandparent(ann, jim)).
3. Search Bias Configuration
To keep the search space finite, we specify the target predicate (the head) and the helper predicates (the body) that Popper is allowed to use to construct candidate rules.
Here is the code in source-code/inductive_logic_popper/bias.pl:
1 head_pred(grandparent, 2).
Python Orchestrator
Popper can be executed as a command-line tool or invoked programmatically inside Python. We construct a script that sets the knowledge-base paths and initiates the solver.
Using a hybrid of Prolog and Python in Popper examples comes down to a deliberate architectural decision: Prolog is the target language and testing engine, but Python is the orchestrator. While the Inductive Logic Programming (ILP) core targets first-order logic, the tool itself is implemented as a modern Python application. The hybrid nature of the examples reflects this clean separation of concerns.
- Python as the Orchestrator (Glue Code) The entire multi-engine loop you evaluated earlier (Generation
Testing Feedback) is written in Python:
- State Management: Python maintains the overall state of the search, handles file I/O, parses command-line arguments, and manages timers.
- Inter-Process Communication: Python acts as the coordinator that calls Clingo (via its Python API or subprocesses) to get a candidate, translates that candidate into a format SWI-Prolog understands, spins up the Prolog interpreter, and parses the results.
- Evaluation Frameworks: In many examples or benchmarks, you will see Python scripts (ilpexp.py or popper.py) managing cross-validation, plotting learning curves, or feeding synthetic data generators into the ILP engine.
- Prolog as the Relational Data and Representation Layer: even though Python runs the show, the data and the learned rules must be expressed in a logic programming syntax. Therefore, the input files for a Popper problem use standard Prolog format:
- Background Knowledge (bk.pl): Relational data (e.g., parent(amy, bob).) or procedural background knowledge definitions are written natively in Prolog because SWI-Prolog will execute them directly during the test phase.
- Examples (exs.pl): Positive and negative training instances are defined as Prolog facts (e.g., pos(grandparent(amy, charlie)).).
- The Output: The ultimate goal of Popper is to output a crisp, human-readable Prolog program.
Here is the code in source-code/inductive_logic_popper/run_popper.py:
1 from popper.util import Settings, format_prog
2 from popper.loop import popper
3
4 def main():
5 print("Running Popper Inductive Logic Programming solver programmatically...")
6 settings = Settings(
7 kbpath='.',
8 bk_file='bk.pl',
9 ex_file='exs.pl',
10 bias_file='bias.pl'
11 )
12 best_prog, best_score = popper(settings)
13
14 print("\n--- Popper Output ---")
15 if best_prog:
16 print("Successfully learned rules:")
17 print(format_prog(best_prog))
18 else:
19 print("No program found that fits the examples.")
20
21 if __name__ == '__main__':
22 main()
Running the Learning Algorithm
Ensure that you have uv installed, then run the python script from the source-code/inductive_logic_popper directory:
1 $ uv run run_popper.py
The program will run the ILP loop, outputting the minimal learned Prolog rule that represents the grandparent definition:
1 2.2s Generating partial hypotheses of size: 2
2 2.2s Generating partial hypotheses of size: 3
3 2.2s ********************
4 2.2s New best hypothesis:
5 2.2s tp:5 fn:0 tn:5 fp:0 size:3
6 2.2s grandparent(V0,V1):- parent(V0,V2),parent(V2,V1).
7 2.2s ********************
8 Running Popper Inductive Logic Programming solver programmatically...
9
10 --- Popper Output ---
11 Successfully learned rules:
12 grandparent(V0,V1):- parent(V0,V2),parent(V2,V1).
Popper successfully discovered that a grandparent relationship between person V0 and V1 holds if V0 is the parent of some intermediate person V2, and V2 is the parent of V1.
Key Design Decisions
Why Popper? Many historical ILP systems (like Progol or Aleph) used search heuristics that were prone to local minima, or struggled to scale. Popper solves this by formulating the search as a Constraint Satisfaction Problem. By using Answer Set Programming (Clingo), Popper leverages advanced conflict-driven clause learning solver technology to prune irrelevant logic programs extremely fast.
Bias and the search space. ILP is not magic: if we do not guide the search, the space of possible Prolog programs is infinite. The bias.pl file is crucial because it limits the search to rules containing only parent/2 predicates in their bodies. If we added male/1 or female/1 to the allowed body predicates, Popper would still find the correct rule, but the search space would expand, requiring more candidates to be evaluated before settling on the minimal program.