Deterministic vs Non-Deterministic in LLM Models
A first-principles breakdown using systems thinking: how LLMs produce deterministic probabilities but non-deterministic outputs, and how to control this in Claude Code.
The Core Mental Model
Think of an LLM as a probability machine:
Probability Distribution: A list of all possible next words/tokens, each with a likelihood score (e.g., "the" = 0.3, "a" = 0.25, "an" = 0.1...).
Sampling: The process of choosing one token from that distribution.
Deterministic Systems
Deterministic: Given the same input, you always get the same output. Like a pure math function: f(2) = 4, every single time.
Key insight: The LLM's internal computation (the neural network forward pass) is deterministic. The probability distribution for the next token is always identical for the same input.
// Conceptual model of deterministic computation
function computeProbabilities(prompt: string): Map<string, number> {
// Neural network math - ALWAYS produces same output for same input
return new Map([
["the", 0.35],
["a", 0.25],
["an", 0.15],
// ... thousands more tokens
]);
}
Non-Deterministic Behavior
Non-Deterministic: Given the same input, you may get different outputs. Like rolling dice.
Here's the key insight: Non-determinism is introduced at the sampling step, not in the model itself.
The Temperature Parameter
Temperature: A number (typically 0-2) that controls how "random" the sampling is. Higher = more creative/random. Lower = more predictable.
// Simplified sampling logic
function sampleNextToken(
probabilities: Map<string, number>,
temperature: number
): string {
if (temperature === 0) {
// DETERMINISTIC: Always pick highest probability token
return getHighestProbToken(probabilities); // "greedy decoding"
} else {
// NON-DETERMINISTIC: Randomly sample based on adjusted probabilities
const adjusted = applyTemperature(probabilities, temperature);
return randomSample(adjusted); // involves randomness!
}
}
Other Sources of Non-Determinism
| Source | What It Is | Impact |
|---|---|---|
| Top-p (nucleus sampling) | Only sample from tokens whose cumulative probability ≤ p | Adds randomness |
| Top-k | Only sample from top k most likely tokens | Adds randomness |
| Random seed | Initial state for random number generator | If fixed → reproducible |
| Hardware/Parallelism | GPU floating-point operations can have tiny variations | Usually negligible |
Seed: A starting number for the random generator. Same seed = same "random" sequence = reproducible non-determinism.
Practical Implications
// Example: Anthropic API call
const response = await anthropic.messages.create({
model: "claude-sonnet-4-20250514",
max_tokens: 100,
temperature: 0, // ← DETERMINISTIC (greedy)
messages: [{ role: "user", content: "What is 2+2?" }]
});
// temperature: 0 → Same answer every time
// temperature: 0.7 → Creative variation
// temperature: 1.5 → High randomness (can be incoherent)
When to Use Each
| Use Case | Temperature | Why |
|---|---|---|
| Code generation | 0 - 0.2 | Correctness matters, creativity doesn't |
| Factual Q&A | 0 | Consistency and accuracy |
| Creative writing | 0.7 - 1.0 | Want variety and surprise |
| Brainstorming | 1.0+ | Maximum divergent thinking |
Deterministic vs Non-Deterministic in Claude Code
Claude Code operates with two fundamentally different control systems:
No randomness, guaranteed execution
Probabilistic - Claude "decides" based on context + sampling
Temperature in Claude Code
Claude Code does not expose temperature settings directly to users. The internal sampling parameters are managed by Anthropic. This means you cannot make Claude Code fully deterministic via temperature=0.
The Deterministic Layer: Hooks System
Hooks: Shell commands that always execute at specific points in Claude's workflow, regardless of what the LLM "thinks."
// .claude/settings.json
{
"hooks": {
"PostToolUse": [
{
// Always run after Claude edits TypeScript files
"matcher": "Edit(*.ts)",
"command": "npx prettier --write $CLAUDE_FILE_PATH && npx eslint --fix $CLAUDE_FILE_PATH"
},
{
// Always typecheck after any .ts modification
"matcher": "Edit(*.ts)",
"command": "npx tsc --noEmit"
}
],
"PreToolUse": [
{
// Block writes to production configs
"matcher": "Write(production.*.json)",
"command": "echo 'BLOCKED: Cannot modify production configs' && exit 1"
}
]
}
}
Why this matters: Hooks execute guaranteed, every time. The LLM cannot "forget" or "decide" to skip them. This is your enforcement mechanism for standards.
The Non-Deterministic Layer: LLM Decisions
Everything the LLM decides is probabilistic:
| What Claude Decides | Deterministic? | Why |
|---|---|---|
| Which files to read | No | LLM interprets context |
| How to structure code | No | Sampling from possibilities |
| Whether to write tests | No | Judgment call |
| Variable/function naming | No | Creative generation |
| Error handling approach | No | Multiple valid solutions |
# Run 1
claude "Create a function to validate email"
# Output might be:
function validateEmail(email: string): boolean {
return /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email);
}
# Run 2 (same prompt)
claude "Create a function to validate email"
# Output might be:
const isValidEmail = (input: string): boolean => {
const emailRegex = /^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/;
return emailRegex.test(input);
};
Both are correct — this is the nature of non-determinism in code generation.
Strategies to Increase Predictability
Since you can't set temperature=0 in Claude Code, here are systems-level approaches:
1. Constrain with CLAUDE.md
<!-- CLAUDE.md in your project root -->
# Code Standards (DETERMINISTIC RULES)
## TypeScript Conventions
- ALWAYS use `function` keyword, never arrow functions for top-level
- ALWAYS use explicit return types
- ALWAYS use `interface` over `type` for object shapes
- Variable naming: camelCase, descriptive, no abbreviations
## File Structure
- One component per file
- Tests in `__tests__/` directory
- Naming: `ComponentName.tsx`, `ComponentName.test.tsx`
## Error Handling
- ALWAYS use custom error classes
- ALWAYS include error codes
- Pattern: try/catch with typed errors
2. Use Hooks for Enforcement
// settings.json - GUARANTEED execution
{
"hooks": {
"PostToolUse": [
{
"matcher": "Edit(*.tsx)",
"command": "npm run lint:fix && npm run typecheck"
}
]
}
}
3. Permission System (Deterministic Boundaries)
{
"permissions": {
"allow": [
"Read",
"Write(src/**)",
"Bash(npm run:*)",
"Bash(git diff)",
"Bash(git status)"
],
"deny": [
"Write(.env*)",
"Write(*.config.js)",
"Bash(rm *)",
"Bash(git push *)",
"Bash(sudo *)"
]
}
}
This is deterministic: Claude cannot execute denied operations, period. No probability involved.
Key Takeaways
| Concept | In Claude Code |
|---|---|
| Temperature | Not user-configurable (feature requested) |
| True Determinism | Hooks + Permissions (shell-level enforcement) |
| Soft Determinism | CLAUDE.md + Custom Commands (constrains options) |
| Non-Determinism | All LLM decisions (code structure, naming, approach) |
- Use hooks to enforce standards (linting, testing, formatting)
- Use permissions to prevent dangerous operations
- Use CLAUDE.md to constrain the solution space
- Accept that code style will vary—focus on correctness via automated checks
This is defense in depth applied to AI-assisted development.