DeepSeek-Prover-V1.5: Teaching Language Models to Prove Theorems with Compiler Feedback
Hook
What if you could train a language model using a compiler as the judge? DeepSeek-Prover-V1.5 treats Lean 4's type checker as an infinite source of ground-truth feedback, eliminating human labeling entirely.
Context
Formal theorem proving has a verification problem that's the inverse of most AI tasks. In computer vision or NLP, you need humans to label whether outputs are correct. But in mathematics, we have proof assistants—compilers that can verify logical correctness with absolute certainty. The challenge isn't checking if a proof is valid; it's generating one in the first place.
Traditional approaches to automated theorem proving relied on classical search algorithms or human-curated tactic databases. Recent neural approaches like GPT-f showed that language models could learn proof patterns from human-written code, but they inherited the biases and limitations of their training data. DeepSeek-Prover-V1.5 represents a different paradigm: use supervised learning to bootstrap basic competence, then let the model discover novel proof strategies through reinforcement learning, with Lean 4's compiler providing immediate, unambiguous feedback on every attempted tactic. The result is a system that proves 63.5% of miniF2F test problems—a 13.5% absolute improvement over its predecessor—by combining neural pattern recognition with symbolic verification.
Technical Insight
The architecture consists of three distinct training stages that progressively specialize the model for theorem proving. It starts with DeepSeekMath-Base (a math-focused language model), applies supervised fine-tuning on curated Lean 4 proofs, then employs reinforcement learning from proof assistant feedback (RLPAF) where the Lean compiler itself generates rewards.
The RLPAF stage is where things get interesting. Unlike standard RLHF which requires human preference labels, RLPAF treats Lean 4 as an oracle. When the model proposes a tactic, the system sends it to Lean via subprocess communication, and the type checker responds with either a new proof state (success) or an error message (failure). This binary feedback—mathematically rigorous and infinitely scalable—drives policy optimization without any human in the loop.
Here's how the training loop interfaces with Lean 4:
# Simplified proof verification loop
class LeanVerifier:
def __init__(self, lean_path, mathlib_path):
self.process = subprocess.Popen(
[lean_path, '--server'],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
cwd=mathlib_path
)
def verify_tactic(self, proof_state, tactic):
# Send current proof state + proposed tactic
request = {
'goals': proof_state.goals,
'hypotheses': proof_state.hypotheses,
'tactic': tactic
}
self.process.stdin.write(json.dumps(request).encode())
# Lean responds with new state or error
response = json.loads(self.process.stdout.readline())
if response['status'] == 'success':
return ProofState(response['new_goals']), 1.0 # reward
else:
return None, -1.0 # terminal failure
The verification happens in real-time during both training and inference. Every tactic the model generates must pass through Lean's type checker before the search continues. If a tactic fails, that branch terminates immediately—there's no partial credit for "almost correct" proofs.
At inference time, DeepSeek-Prover-V1.5 uses RMaxTS (Monte-Carlo tree search with intrinsic rewards) rather than simple beam search. Standard MCTS struggles in theorem proving because the reward signal is sparse—you only get positive feedback when the entire proof completes. RMaxTS addresses this by adding intrinsic reward bonuses for exploring novel proof states:
class RMaxTS:
def __init__(self, model, verifier, exploration_bonus=1.0):
self.model = model
self.verifier = verifier
self.visit_counts = defaultdict(int)
self.exploration_bonus = exploration_bonus
def select_action(self, state):
# Generate candidate tactics from model
tactics = self.model.generate(state, num_samples=32)
best_score = float('-inf')
best_tactic = None
for tactic in tactics:
# Intrinsic reward for state novelty
state_hash = hash(str(state) + tactic)
intrinsic_reward = self.exploration_bonus / (1 + self.visit_counts[state_hash])
# Model confidence + novelty bonus
score = tactic.log_prob + intrinsic_reward
if score > best_score:
best_score = score
best_tactic = tactic
return best_tactic
The intrinsic reward mechanism is critical. Without it, the search quickly collapses into repeatedly trying high-probability tactics that the model is confident about but which may not lead to valid proofs. By incentivizing exploration of states the model hasn't visited frequently, RMaxTS discovers alternative proof paths that supervised learning alone wouldn't find.
The training progression reveals something surprising: performance goes 42.2% (base) → 57.4% (SFT) → 60.2% (RL) on miniF2F. The non-monotonic jump suggests that supervised fine-tuning causes the model to overfit on human proof patterns—the specific tactics and strategies that mathematicians tend to use. Reinforcement learning then recovers performance by discovering alternative valid proofs that humans might not write but which Lean accepts. The model learns to exploit the gap between "how humans prove theorems" and "what the type checker accepts."
One architectural constraint shapes everything: Lean 4 state must serialize to text. The model can't directly manipulate abstract syntax trees or proof objects. Instead, each proof state—goals, hypotheses, metavariables—gets formatted as a string that the transformer processes through its attention mechanism. This means the model learns to "read" formal mathematics in Lean's concrete syntax rather than operating on symbolic representations directly. It's both a limitation (no direct structural reasoning) and a strength (standard transformer architectures work without modification).
Gotcha
The operational complexity is substantial. Before you can run any inference, you need to install the full Lean 4 toolchain, clone mathlib4 as a submodule, and run lake build to compile the entire standard library. This isn't a pip-install-and-go situation—expect several gigabytes of downloads and hours of compilation time on first setup. If mathlib4 updates, you may need to rebuild everything. This makes DeepSeek-Prover-V1.5 impractical for deployment scenarios where users expect quick installation.
The performance cliff between problem sets is stark and revealing. While the model achieves 63.5% on miniF2F (high school competition math), it drops to 25.3% on ProofNet (undergraduate-level theorems). This isn't a small gap—it suggests the model relies heavily on pattern matching against training data rather than developing robust compositional reasoning. When problems require chaining multiple complex definitions or working with abstract algebraic structures beyond its training distribution, performance collapses. The system also generates whole proofs in single forward passes rather than interactive step-by-step refinement, meaning if any tactic in a 20-step proof fails, all prior work is discarded. There's no proof repair mechanism, no way to partially credit "almost correct" attempts, and no backtracking to fix specific failed tactics while preserving the rest of the proof structure.
Verdict
Use if: You're researching formal theorem proving with access to significant compute resources, need to automate formalization of high-school or early undergraduate mathematics, or you're building neurosymbolic systems where ground-truth verifiers exist and you want to see how RLPAF works in practice. The approach is architecturally significant for anyone exploring how compilers and type checkers can serve as reward functions for RL. Skip if: You need production deployment with reasonable setup time, want interactive proof assistants for working mathematicians (Lean Copilot or GPT-f offer better UX), work on informal mathematical reasoning without formal specifications, or need systems that generalize to research-level mathematics. The Lean 4 dependency and multi-hour build process make this strictly a research tool, not production infrastructure.