> your AI agent picks dependencies from memory; give it dated facts — try starlog.dev ↗ vet your agent's deps ↗ vibe-coding is fine. vibe-importing isn’t. — try starlog.dev ↗ vibe-importing isn’t fine ↗ your agent has never seen your private packages — try starlog.dev ↗ facts for private packages ↗ a linter for the dependencies your AI agent picks — try starlog.dev ↗ a linter for agent deps ↗ whois is redacted, cdns mask the rest — get the real operator — try whoisgeni.us ↗ who really runs that domain ↗ domain attribution that shows its work — full evidence chain — try whoisgeni.us ↗ domain intel w/ evidence ↗

← Back to Articles

DeepSeek-Prover-V1.5: Teaching Language Models to Prove Theorems with Compiler Feedback

[ View on GitHub ]

DeepSeek-Prover-V1.5: Teaching Language Models to Prove Theorems with Compiler Feedback

Hook

What if you could train a language model using a compiler as the judge? DeepSeek-Prover-V1.5 treats Lean 4's type checker as an infinite source of ground-truth feedback, eliminating human labeling entirely.

Context

Formal theorem proving has a verification problem that's the inverse of most AI tasks. In computer vision or NLP, you need humans to label whether outputs are correct. But in mathematics, we have proof assistants—compilers that can verify logical correctness with absolute certainty. The challenge isn't checking if a proof is valid; it's generating one in the first place.

Traditional approaches to automated theorem proving relied on classical search algorithms or human-curated tactic databases. Recent neural approaches like GPT-f showed that language models could learn proof patterns from human-written code, but they inherited the biases and limitations of their training data. DeepSeek-Prover-V1.5 represents a different paradigm: use supervised learning to bootstrap basic competence, then let the model discover novel proof strategies through reinforcement learning, with Lean 4's compiler providing immediate, unambiguous feedback on every attempted tactic. The result is a system that proves 63.5% of miniF2F test problems—a 13.5% absolute improvement over its predecessor—by combining neural pattern recognition with symbolic verification.

Technical Insight

Inference

Training

Pre-trained weights

Curated Lean proofs

Proposed tactic

Success: new proof state

Failure: error

Policy updates

Tactic generation

Candidate tactic

Verification result

Intrinsic rewards

guide exploration

Complete proof

DeepSeekMath-Base Model

Supervised Fine-Tuning

RLPAF Training Loop

RMaxTS Search Engine

Lean 4 Type Checker

Specialized Prover Model

Verified Theorem

System architecture — auto-generated

The architecture consists of three distinct training stages that progressively specialize the model for theorem proving. It starts with DeepSeekMath-Base (a math-focused language model), applies supervised fine-tuning on curated Lean 4 proofs, then employs reinforcement learning from proof assistant feedback (RLPAF) where the Lean compiler itself generates rewards.

The RLPAF stage is where things get interesting. Unlike standard RLHF which requires human preference labels, RLPAF treats Lean 4 as an oracle. When the model proposes a tactic, the system sends it to Lean via subprocess communication, and the type checker responds with either a new proof state (success) or an error message (failure). This binary feedback—mathematically rigorous and infinitely scalable—drives policy optimization without any human in the loop.

Here's how the training loop interfaces with Lean 4:

# Simplified proof verification loop
class LeanVerifier:
    def __init__(self, lean_path, mathlib_path):
        self.process = subprocess.Popen(
            [lean_path, '--server'],
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            cwd=mathlib_path
        )
    
    def verify_tactic(self, proof_state, tactic):
        # Send current proof state + proposed tactic
        request = {
            'goals': proof_state.goals,
            'hypotheses': proof_state.hypotheses,
            'tactic': tactic
        }
        self.process.stdin.write(json.dumps(request).encode())
        
        # Lean responds with new state or error
        response = json.loads(self.process.stdout.readline())
        
        if response['status'] == 'success':
            return ProofState(response['new_goals']), 1.0  # reward
        else:
            return None, -1.0  # terminal failure

The verification happens in real-time during both training and inference. Every tactic the model generates must pass through Lean's type checker before the search continues. If a tactic fails, that branch terminates immediately—there's no partial credit for "almost correct" proofs.

At inference time, DeepSeek-Prover-V1.5 uses RMaxTS (Monte-Carlo tree search with intrinsic rewards) rather than simple beam search. Standard MCTS struggles in theorem proving because the reward signal is sparse—you only get positive feedback when the entire proof completes. RMaxTS addresses this by adding intrinsic reward bonuses for exploring novel proof states:

class RMaxTS:
    def __init__(self, model, verifier, exploration_bonus=1.0):
        self.model = model
        self.verifier = verifier
        self.visit_counts = defaultdict(int)
        self.exploration_bonus = exploration_bonus
    
    def select_action(self, state):
        # Generate candidate tactics from model
        tactics = self.model.generate(state, num_samples=32)
        
        best_score = float('-inf')
        best_tactic = None
        
        for tactic in tactics:
            # Intrinsic reward for state novelty
            state_hash = hash(str(state) + tactic)
            intrinsic_reward = self.exploration_bonus / (1 + self.visit_counts[state_hash])
            
            # Model confidence + novelty bonus
            score = tactic.log_prob + intrinsic_reward
            
            if score > best_score:
                best_score = score
                best_tactic = tactic
        
        return best_tactic

The intrinsic reward mechanism is critical. Without it, the search quickly collapses into repeatedly trying high-probability tactics that the model is confident about but which may not lead to valid proofs. By incentivizing exploration of states the model hasn't visited frequently, RMaxTS discovers alternative proof paths that supervised learning alone wouldn't find.

The training progression reveals something surprising: performance goes 42.2% (base) → 57.4% (SFT) → 60.2% (RL) on miniF2F. The non-monotonic jump suggests that supervised fine-tuning causes the model to overfit on human proof patterns—the specific tactics and strategies that mathematicians tend to use. Reinforcement learning then recovers performance by discovering alternative valid proofs that humans might not write but which Lean accepts. The model learns to exploit the gap between "how humans prove theorems" and "what the type checker accepts."

One architectural constraint shapes everything: Lean 4 state must serialize to text. The model can't directly manipulate abstract syntax trees or proof objects. Instead, each proof state—goals, hypotheses, metavariables—gets formatted as a string that the transformer processes through its attention mechanism. This means the model learns to "read" formal mathematics in Lean's concrete syntax rather than operating on symbolic representations directly. It's both a limitation (no direct structural reasoning) and a strength (standard transformer architectures work without modification).

Gotcha

The operational complexity is substantial. Before you can run any inference, you need to install the full Lean 4 toolchain, clone mathlib4 as a submodule, and run lake build to compile the entire standard library. This isn't a pip-install-and-go situation—expect several gigabytes of downloads and hours of compilation time on first setup. If mathlib4 updates, you may need to rebuild everything. This makes DeepSeek-Prover-V1.5 impractical for deployment scenarios where users expect quick installation.

The performance cliff between problem sets is stark and revealing. While the model achieves 63.5% on miniF2F (high school competition math), it drops to 25.3% on ProofNet (undergraduate-level theorems). This isn't a small gap—it suggests the model relies heavily on pattern matching against training data rather than developing robust compositional reasoning. When problems require chaining multiple complex definitions or working with abstract algebraic structures beyond its training distribution, performance collapses. The system also generates whole proofs in single forward passes rather than interactive step-by-step refinement, meaning if any tactic in a 20-step proof fails, all prior work is discarded. There's no proof repair mechanism, no way to partially credit "almost correct" attempts, and no backtracking to fix specific failed tactics while preserving the rest of the proof structure.

Verdict

Use if: You're researching formal theorem proving with access to significant compute resources, need to automate formalization of high-school or early undergraduate mathematics, or you're building neurosymbolic systems where ground-truth verifiers exist and you want to see how RLPAF works in practice. The approach is architecturally significant for anyone exploring how compilers and type checkers can serve as reward functions for RL. Skip if: You need production deployment with reasonable setup time, want interactive proof assistants for working mathematicians (Lean Copilot or GPT-f offer better UX), work on informal mathematical reasoning without formal specifications, or need systems that generalize to research-level mathematics. The Lean 4 dependency and multi-hour build process make this strictly a research tool, not production infrastructure.