Research
Critical rules live in COBOL; unverified LLM output is not enough.
Legacy Rule Extractor
Extract business rules from COBOL, score against a hand-written answer key, replay behaviour until it matches.
- Python
- COBOL
- LLMs
- Evaluation
Context
Legacy COBOL systems hide critical business rules in procedural code. Teams need those rules explicit — without trusting unverified LLM output.
Approach
Build an agentic extraction loop: plan the modules to inspect, run LLM extractors, score against a hand-written answer key, then replay behaviour.
Solution
Python evaluation harness + structured prompts. Failures are first-class cases. The loop adapts prompts and chunking until behaviour matches.
Evidence
Research experiment focused on measurable agreement with a human answer key — not demo-only chat transcripts.