Deterministic vs Non-Deterministic in LLM Models

A first-principles breakdown using systems thinking: how LLMs produce deterministic probabilities but non-deterministic outputs, and how to control this in Claude Code.


The Core Mental Model

Think of an LLM as a probability machine:

Input (prompt) LLM Probability Distribution Sampling Output (token)
Probability Distribution: A list of all possible next words/tokens, each with a likelihood score (e.g., "the" = 0.3, "a" = 0.25, "an" = 0.1...).
Sampling: The process of choosing one token from that distribution.

Deterministic Systems

Deterministic: Given the same input, you always get the same output. Like a pure math function: f(2) = 4, every single time.

Key insight: The LLM's internal computation (the neural network forward pass) is deterministic. The probability distribution for the next token is always identical for the same input.

TypeScript
// Conceptual model of deterministic computation
function computeProbabilities(prompt: string): Map<string, number> {
  // Neural network math - ALWAYS produces same output for same input
  return new Map([
    ["the", 0.35],
    ["a", 0.25],
    ["an", 0.15],
    // ... thousands more tokens
  ]);
}

Non-Deterministic Behavior

Non-Deterministic: Given the same input, you may get different outputs. Like rolling dice.

Here's the key insight: Non-determinism is introduced at the sampling step, not in the model itself.

The Temperature Parameter

Temperature: A number (typically 0-2) that controls how "random" the sampling is. Higher = more creative/random. Lower = more predictable.
TypeScript
// Simplified sampling logic
function sampleNextToken(
  probabilities: Map<string, number>,
  temperature: number
): string {
  if (temperature === 0) {
    // DETERMINISTIC: Always pick highest probability token
    return getHighestProbToken(probabilities); // "greedy decoding"
  } else {
    // NON-DETERMINISTIC: Randomly sample based on adjusted probabilities
    const adjusted = applyTemperature(probabilities, temperature);
    return randomSample(adjusted); // involves randomness!
  }
}
✅ Deterministic Zone
Prompt LLM Model Probabilities
🎲 Non-Deterministic Zone (if temp > 0)
Probabilities Sampling (+ random) Output Token

Other Sources of Non-Determinism

Source What It Is Impact
Top-p (nucleus sampling) Only sample from tokens whose cumulative probability ≤ p Adds randomness
Top-k Only sample from top k most likely tokens Adds randomness
Random seed Initial state for random number generator If fixed → reproducible
Hardware/Parallelism GPU floating-point operations can have tiny variations Usually negligible
Seed: A starting number for the random generator. Same seed = same "random" sequence = reproducible non-determinism.

Practical Implications

TypeScript
// Example: Anthropic API call
const response = await anthropic.messages.create({
  model: "claude-sonnet-4-20250514",
  max_tokens: 100,
  temperature: 0,        // ← DETERMINISTIC (greedy)
  messages: [{ role: "user", content: "What is 2+2?" }]
});

// temperature: 0   → Same answer every time
// temperature: 0.7 → Creative variation
// temperature: 1.5 → High randomness (can be incoherent)

When to Use Each

Use Case Temperature Why
Code generation 0 - 0.2 Correctness matters, creativity doesn't
Factual Q&A 0 Consistency and accuracy
Creative writing 0.7 - 1.0 Want variety and surprise
Brainstorming 1.0+ Maximum divergent thinking
Key Takeaway: The LLM is a deterministic calculator of probabilities. Non-determinism is a choice we inject via sampling parameters to get creative, varied outputs.

Deterministic vs Non-Deterministic in Claude Code

Claude Code operates with two fundamentally different control systems:

Deterministic Layer - You Control 100%
Hooks (Always Execute) Scripts (Shell) Permission Rules (Allow/Deny)

No randomness, guaranteed execution

Non-Deterministic Layer - LLM Decides
LLM Decision Making Code Writing Style Tool Choice

Probabilistic - Claude "decides" based on context + sampling

Critical Insight: Must the action ALWAYS happen, regardless of Claude's judgment? If YES → Use Hook (deterministic). If NO → Use Prompt (probabilistic).

Temperature in Claude Code

Claude Code does not expose temperature settings directly to users. The internal sampling parameters are managed by Anthropic. This means you cannot make Claude Code fully deterministic via temperature=0.

The Deterministic Layer: Hooks System

Hooks: Shell commands that always execute at specific points in Claude's workflow, regardless of what the LLM "thinks."
User Request
Natural language input
PreToolUse Hook
Always runs BEFORE Claude uses any tool
Deterministic
Claude's Tool Use (Edit, Bash...)
LLM decides what/how to execute
Non-Deterministic
PostToolUse Hook
Always runs AFTER Claude's tool completes
Deterministic
JSON
// .claude/settings.json
{
  "hooks": {
    "PostToolUse": [
      {
        // Always run after Claude edits TypeScript files
        "matcher": "Edit(*.ts)",
        "command": "npx prettier --write $CLAUDE_FILE_PATH && npx eslint --fix $CLAUDE_FILE_PATH"
      },
      {
        // Always typecheck after any .ts modification
        "matcher": "Edit(*.ts)",
        "command": "npx tsc --noEmit"
      }
    ],
    "PreToolUse": [
      {
        // Block writes to production configs
        "matcher": "Write(production.*.json)",
        "command": "echo 'BLOCKED: Cannot modify production configs' && exit 1"
      }
    ]
  }
}

Why this matters: Hooks execute guaranteed, every time. The LLM cannot "forget" or "decide" to skip them. This is your enforcement mechanism for standards.

The Non-Deterministic Layer: LLM Decisions

Everything the LLM decides is probabilistic:

What Claude Decides Deterministic? Why
Which files to read No LLM interprets context
How to structure code No Sampling from possibilities
Whether to write tests No Judgment call
Variable/function naming No Creative generation
Error handling approach No Multiple valid solutions
Bash
# Run 1
claude "Create a function to validate email"

# Output might be:
function validateEmail(email: string): boolean {
  return /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email);
}

# Run 2 (same prompt)
claude "Create a function to validate email"

# Output might be:
const isValidEmail = (input: string): boolean => {
  const emailRegex = /^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/;
  return emailRegex.test(input);
};

Both are correct — this is the nature of non-determinism in code generation.


Strategies to Increase Predictability

Since you can't set temperature=0 in Claude Code, here are systems-level approaches:

1. Constrain with CLAUDE.md

Markdown
<!-- CLAUDE.md in your project root -->
# Code Standards (DETERMINISTIC RULES)

## TypeScript Conventions
- ALWAYS use `function` keyword, never arrow functions for top-level
- ALWAYS use explicit return types
- ALWAYS use `interface` over `type` for object shapes
- Variable naming: camelCase, descriptive, no abbreviations

## File Structure
- One component per file
- Tests in `__tests__/` directory
- Naming: `ComponentName.tsx`, `ComponentName.test.tsx`

## Error Handling
- ALWAYS use custom error classes
- ALWAYS include error codes
- Pattern: try/catch with typed errors
Insight: More specific instructions → Less variance in outputs. You're constraining the probability distribution by narrowing valid choices.

2. Use Hooks for Enforcement

JSON
// settings.json - GUARANTEED execution
{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Edit(*.tsx)",
        "command": "npm run lint:fix && npm run typecheck"
      }
    ]
  }
}

3. Permission System (Deterministic Boundaries)

JSON
{
  "permissions": {
    "allow": [
      "Read",
      "Write(src/**)",
      "Bash(npm run:*)",
      "Bash(git diff)",
      "Bash(git status)"
    ],
    "deny": [
      "Write(.env*)",
      "Write(*.config.js)",
      "Bash(rm *)",
      "Bash(git push *)",
      "Bash(sudo *)"
    ]
  }
}

This is deterministic: Claude cannot execute denied operations, period. No probability involved.

← Fully Deterministic Fully Non-Deterministic →
Hooks
Shell commands
100% control
Permissions
Allow/Deny
100% control
CLAUDE.md
Instructions
~80% control
Free Prompts
Natural language
~50% control

Key Takeaways

Concept In Claude Code
Temperature Not user-configurable (feature requested)
True Determinism Hooks + Permissions (shell-level enforcement)
Soft Determinism CLAUDE.md + Custom Commands (constrains options)
Non-Determinism All LLM decisions (code structure, naming, approach)
The Engineering Mindset: Instead of trying to make the LLM deterministic, build deterministic guardrails around it:
  • Use hooks to enforce standards (linting, testing, formatting)
  • Use permissions to prevent dangerous operations
  • Use CLAUDE.md to constrain the solution space
  • Accept that code style will vary—focus on correctness via automated checks

This is defense in depth applied to AI-assisted development.