LLM Logic Guardrails
Large Language Models (LLMs) are incredibly capable when it comes to open-ended generation, creative writing, and basic translation. However, they suffer from deep structural limitations that make them risky for high-stakes business logic:
- Hallucinations: They generate plausible-sounding but entirely fabricated facts.
- Inability to guarantee constraints: An LLM cannot be mathematically guaranteed to follow instructions or comply with safety bounds.
- Arithmetic errors: They often struggle with exact math or calculating cumulative percentages.
In critical domains like financial planning, computational law, or medical diagnosis, we cannot display raw LLM output to end users. We need a way to filter, validate, and verify the model’s outputs.
This chapter presents a neuro-symbolic pattern that uses a symbolic logic engine (Prolog) as a strict compliance guardrail on top of a neural model (the LLM).
The Neuro-Symbolic Guardrail Pattern
The architecture operates as a pipeline:
- Prompt & Structure: The user inputs a prompt. The LLM is instructed to return its recommendation in a structured format, such as JSON.
- Parsing & Bridge: The application parses the LLM’s JSON output and passes it across a bidirectional bridge (Janus) into a SWI-Prolog environment.
- Prolog Constraints: A predefined Prolog policy database contains strict rules (such as arithmetic checks, age-based risk limits, and exclusions).
- Validation Query: Prolog evaluates the recommendation. It returns an empty list if the recommendation is valid, or a list of specific error explanations if any constraints are violated.
- Action: If valid, the recommendation is shown to the user. If invalid, the system rejects it or feeds the error descriptions back to the LLM to request a corrected response (self-repair loop).
This design ensures that even if the LLM attempts to recommend high-risk assets to a senior citizen or creates a portfolio that doesn’t sum to 100%, the symbolic engine mathematically prevents it from reaching the user.
Prolog Guardrail Rules
We define the constraints using a Prolog module that parses the JSON string into a dict, and evaluates it against multiple rules using findall/3.
Here is the implementation in source-code/llm_logic_guardrails/prolog/guardrails.pl:
1 :- module(guardrails, [
2 validate_portfolio_json/2
3 ]).
4
5 :- use_module(library(http/json)).
6
7 /** <module> Financial Guardrails Module
8 *
9 * Validates investment recommendations generated by LLMs using
10 symbolic rules.
11 * Input is a JSON string representing the recommendation:
12 * {
13 * "client_age": 70,
14 * "risk_tolerance": "low",
15 * "allocations": {
16 * "stocks": 20,
17 * "bonds": 50,
18 * "crypto": 10,
19 * "cash": 20
20 * }
21 * }
22 */
23
24 % Main entry point. Parses JSON and checks all policy rules.
25 % Returns a list of error atoms. If empty, the portfolio is valid.
26 validate_portfolio_json(JsonString, Errors) :-
27 setup_call_cleanup(
28 open_string(JsonString, Stream),
29 json_read_dict(Stream, Dict),
30 close(Stream)
31 ),
32 findall(Error, check_policy(Dict, Error), Errors).
33
34 % --- Policy Rules ---
35
36 % 1. Total sum must be exactly 100%
37 check_policy(Dict, "Total allocation must sum to exactly 100%") :-
38 get_allocations(Dict, Stocks, Bonds, Crypto, Cash),
39 Total is Stocks + Bonds + Crypto + Cash,
40 Total \= 100.
41
42 % 2. If client is senior (> 65), high-risk assets (stocks + crypto) must
43 % be <= 30%
44 check_policy(Dict, Error) :-
45 Age = Dict.get(client_age),
46 Age > 65,
47 get_allocations(Dict, Stocks, _, Crypto, _),
48 HighRiskAllocation is Stocks + Crypto,
49 HighRiskAllocation > 30,
50 format(string(Error),
51 "Senior client (age ~w) has ~w% in high-risk assets (max: 30%)",
52 [Age, HighRiskAllocation]).
53
54 % 3. If risk tolerance is 'low', crypto allocation must be 0%
55 check_policy(Dict,
56 "Low risk tolerance portfolio cannot contain speculative crypto assets") :-
57 Risk = Dict.get(risk_tolerance),
58 Risk == "low", % json_read_dict parses strings as SWI-Prolog string
59 % terms or atoms depending on settings
60 get_allocations(Dict, _, _, Crypto, _),
61 Crypto > 0.
62
63 % 4. If risk tolerance is 'low', conservative assets (bonds + cash) must
64 % be >= 50%
65 check_policy(Dict, Error) :-
66 Risk = Dict.get(risk_tolerance),
67 Risk == "low",
68 get_allocations(Dict, _, Bonds, _, Cash),
69 Conservative is Bonds + Cash,
70 Conservative < 50,
71 format(string(Error),
72 "Low risk tolerance requires at least 50% in conservative assets (currently ~w%)", [Conservative]).
73
74 % 5. No asset allocation can be negative
75 check_policy(Dict, "Asset allocations cannot be negative") :-
76 get_allocations(Dict, Stocks, Bonds, Crypto, Cash),
77 (Stocks < 0 ; Bonds < 0 ; Crypto < 0 ; Cash < 0).
78
79 % --- Helper to extract allocations safely with default 0 if missing ---
80 get_allocations(Dict, Stocks, Bonds, Crypto, Cash) :-
81 Allocations = Dict.get(allocations),
82 Stocks = Allocations.get(stocks, 0),
83 Bonds = Allocations.get(bonds, 0),
Python Verification Harness
We use the Janus Python-Prolog bridge to load the Prolog rules and run our validations programmatically.
Here is the code in source-code/llm_logic_guardrails/verify_llm.py:
1 import janus_swi as janus
2
3 # Define simulated LLM portfolio recommendations
4 VALID_RECOMMENDATION = {
5 "client_age": 70,
6 "risk_tolerance": "low",
7 "allocations": {
8 "stocks": 10,
9 "bonds": 60,
10 "crypto": 0,
11 "cash": 30
12 }
13 }
14
15 INVALID_RECOMMENDATION = {
16 "client_age": 72,
17 "risk_tolerance": "low",
18 "allocations": {
19 "stocks": 40, # Violates: High-risk stocks + crypto (50%) > 30% for age > 65
20 "bonds": 30,
21 "crypto": 10, # Violates: Low risk tolerance cannot have crypto
22 "cash": 10 # Violates: Bonds + Cash (40%) < 50% for low risk
23 } # Violates: Sum is 40 + 30 + 10 + 10 = 90% (not 100%)
24 }
25
26 def test_recommendation(name, recommendation_dict):
27 print(f"\nTesting recommendation: {name}")
28 print("LLM Output JSON:")
29 json_str = json.dumps(recommendation_dict, indent=2)
30 print(json_str)
31
32 # Query Prolog guardrails
33 query_str = "validate_portfolio_json(Json, Errors)"
34 res = janus.query_once(query_str, {"Json": json_str})
35
36 errors = res["Errors"]
37 if not errors:
38 print("✅ Guardrail Check Passed: Recommendation is SAFE.")
39 else:
40 print("❌ Guardrail Check Failed! Violations found:")
41 for err in errors:
42 # Decode bytes to string if returned as bytes
43 err_str = err.decode('utf-8') if isinstance(err, bytes) else str(err)
44 print(f" - {err_str}")
45
46 def main():
47 print("Consulting Prolog guardrail rules...")
48 janus.consult("prolog/guardrails.pl")
49
50 # Test valid case
51 test_recommendation("Valid Senior Low-Risk Portfolio", VALID_RECOMMENDATION)
52
53 # Test invalid case
54 test_recommendation("Invalid Senior Low-Risk Portfolio", INVALID_RECOMMENDATION)
55
56 if __name__ == '__main__':
57 main()
Running the Verification Script
You can execute the verification harness using uv. Navigate to source-code/llm_logic_guardrails and run:
1 $ uv run verify_llm.py
The console output demonstrates how the valid portfolio passes silently, whereas the invalid portfolio produces a clear list of the exact constraints violated:
1 Consulting Prolog guardrail rules...
2
3 Testing recommendation: Valid Senior Low-Risk Portfolio
4 LLM Output JSON:
5 {
6 "client_age": 70,
7 "risk_tolerance": "low",
8 "allocations": {
9 "stocks": 10,
10 "bonds": 60,
11 "crypto": 0,
12 "cash": 30
13 }
14 }
15 ✅ Guardrail Check Passed: Recommendation is SAFE.
16
17 Testing recommendation: Invalid Senior Low-Risk Portfolio
18 LLM Output JSON:
19 {
20 "client_age": 72,
21 "risk_tolerance": "low",
22 "allocations": {
23 "stocks": 40,
24 "bonds": 30,
25 "crypto": 10,
26 "cash": 10
27 }
28 }
29 ❌ Guardrail Check Failed! Violations found:
30 - Total allocation must sum to exactly 100%
31 - Senior client (age 72) has 50% in high-risk assets (max: 30%)
32 - Low risk tolerance portfolio cannot contain speculative crypto assets
33 - Low risk tolerance requires at least 50% in conservative assets (currently 40%)
Key Design Decisions
Why use Janus and Python instead of writing everything in Prolog? While SWI-Prolog can handle network requests, parsing, and LLM integrations directly, Python has a much wider ecosystem of libraries for building production-grade LLM applications (such as LangChain, LlamaIndex, and the official Google GenAI SDK). Janus offers an ideal compromise: we can write our API plumbing and LLM pipelines in Python, but delegate the critical verification steps to Prolog, keeping our policy logic clean and declarative.
Using findall/3 for complete error lists. In typical Prolog programs, we rely on backtracking to find a solution. If a validation failed, Prolog would normally return false on the first rule violation and halt. However, when returning errors to a user or feeding them back to an LLM for correction, we want all violations reported at once. Using findall(Error, check_policy(Dict, Error), Errors) forces the engine to evaluate every safety check and collect all generated error strings into a single list.