Duncan Leung
The LLM Dumb Zone: Why Your AI Gets Worse Before It Runs Out of Context
Published on

The LLM Dumb Zone: Why Your AI Gets Worse Before It Runs Out of Context

Authors

Your LLM doesn't suddenly break when the context window fills up. It gets gradually dumber long before that - and by the time you notice, you've been working in the degradation zone for a while.

I was watching my Claude Code statusline tick down from ctx 85% to ctx 40% and realized the number was answering the wrong question. "How much context is left?" is about when the conversation dies. The question I actually cared about was: how many tokens have I consumed, and am I past the point where quality degrades?

Those are different questions. And it turns out percentages are the wrong unit entirely: a 1M-context model at 40% has consumed 400K tokens and is deep into the degradation zone, while a 200K model at 40% has consumed only 80K tokens and is fine. The attention mechanism cares about absolute distance, not relative fill.

The Research: Context Rot Is Real and Measurable

The core finding comes from three converging lines of research:

Lost in the Middle (Stanford, 2023)

The original paper that named the problem1. When you give an LLM a set of documents and ask it to find information, its accuracy follows a U-shaped curve based on where the relevant information sits:

Accuracy
  |
  |  ██                                        ██
  |  ████                                    ████
  |  ██████                                ██████
  |  ████████                            ████████
  |  ██████████                        ██████████
  |  ████████████                    ████████████
  |  ██████████████              ████████████████
  |  ████████████████        ████████████████████
  |  ██████████████████  ████████████████████████
  +--------------------------------------------------
     Start          Middle              End
                Position of relevant information

Information at the start and end of context gets full attention. Information in the middle can see 30%+ accuracy drops. The model isn't forgetting it exists - the attention mechanism physically under-weights it2.

This isn't a bug in one model. The Stanford team replicated it across six model families. It's a property of the transformer attention mechanism itself.

Context Rot (Chroma, 2025)

Chroma tested 18 state-of-the-art models on a straightforward task: find information in a context window at different fill levels. Every single model got worse as context length increased. Not some models - all of them.

For 1M-token context window models, the observable effect typically kicks in somewhere around 300K-400K tokens. The degradation is gradual and continuous: it worsens at every length increment, not as a sudden cliff. But the slope steepens: the first 100K tokens past the threshold cost you less quality than the next 100K.

Intelligence Degradation Thresholds (2025-2026)

Multiple research groups have converged on similar findings, and the critical detail is that the thresholds are absolute, not relative. A study testing models at different context utilization levels found significant F1 degradation (as much as 45.5%) as token counts grew past certain absolute boundaries. Crucially, the same percentage of context produced wildly different quality depending on the model's window size.

This makes sense architecturally. RoPE (Rotary Position Embedding) decay reduces dot-product similarity between distant token pairs. The decay is a function of the distance between tokens: the absolute positional gap, not what fraction of the window that gap represents. A model that's consumed 400K tokens has 400K tokens of positional distance to attend over, whether its window is 500K or 1M.

From the research, the practical thresholds shake out to:

  • Below ~100K tokens used: Stable performance. Models maintain strong F1 scores and full reasoning fidelity regardless of window size.
  • 100K-200K tokens used: Quality is noticeably fading. Attention to early context weakens, and the model starts missing constraints established early in the conversation.
  • Above ~200K tokens used: The model is materially degraded. It skips instructions, loses track of earlier constraints, and starts repeating itself.

The key insight: a 1M-context model at 40% full has consumed 400K tokens, well into the degraded zone. A 200K model at 40% has consumed only 80K tokens, solidly safe. Percentage-based thresholds treat these as equivalent. They're not.

What Actually Degrades

Context degradation isn't "the model gets dumber at everything." Specific capabilities fail in a predictable order:

First to go (100K-200K tokens):

  • Recall of constraints or decisions from early in the conversation
  • Tracking multiple parallel threads simultaneously
  • Noticing contradictions between something said 200 messages ago and the current request
  • Following complex multi-step instructions that reference earlier context

Last to go (200K+ tokens):

  • Understanding the current message (the model still reads your latest prompt at full fidelity)
  • Following instructions in the system prompt (it sits at the edge of context, where attention is strongest)
  • Basic reasoning on self-contained questions
  • Code generation from a clear, local specification

This is why the degradation is insidious. The model still sounds competent - it's answering your latest question fine. But it's lost the thread of what you decided three refactoring steps ago, or it's forgotten a constraint you set at the start of the session. You don't notice until you review its work and find it contradicted an earlier decision.

Three Zones, Not a Percentage

The key realization: thresholds should be absolute token counts, not percentages. A percentage-based approach treats "40% of 1M tokens" (400K tokens consumed) the same as "40% of 200K tokens" (80K tokens consumed). But the attention mechanism doesn't care about percentages; it cares about the raw distance between tokens. 400K tokens of positional distance degrades quality regardless of how big the window is.

The zones are defined by absolute token thresholds:

  Tokens Used

  0K          100K             200K       300K          700K       window max
  |============|================|==========|==============|==========|
  |   SMART    |     WARN       | DUMB     | DUMB         | DUMB     |
  |            |                | +0-100K  | +100K-500K   | +500K+   |
  |  Full      |  Quality       |          |              |          |
  |  reasoning |  fading.       | Finish   | /compact     | /compact |
  |  fidelity. |  Finish        | task,    | soon.        | NOW or   |
  |  Work      |  current work, | then     |              | new      |
  |  freely.   |  then compact. | compact. |              | session. |
  |============|================|==========|==============|==========|
     (green)      (yellow)       (green)     (yellow)       (red)

SMART (< 100K tokens used): The model has full access to everything in context. Start complex multi-step tasks, set up long-running investigations, establish constraints and conventions. This is where you do your hardest thinking together.

WARN (100K-200K tokens used): Quality is fading but the model is still functional for the current task. Recent context (last 20-30 exchanges) is fine. Early context is getting unreliable. This is your signal to finish the current line of work and run /compact to compress the conversation back toward the SMART zone. Don't start a brand-new complex investigation here.

DUMB (> 200K tokens used): The model is past the quality threshold, but this is a slope, not a cliff. The Stanford study showed a gradual U-shaped curve, not a step function, and the Chroma study found degradation at every length increment. Crossing 200K doesn't mean everything breaks; it means you've entered a zone where quality is measurably worse and continues to worsen with every additional token. How deep into the DUMB zone you are matters:

  • DUMB +0 to +100K: You just crossed the threshold. Finish your current task, then /compact. The model still handles recent context well; it's early context recall that's suffering.
  • DUMB +100K to +500K: You've been degraded for a while. The middle of your conversation is increasingly invisible to the model. /compact soon.
  • DUMB +500K+: Deep degradation. Most of your early context is effectively gone. /compact now or start a new session.

The key insight for WARN zone: /compact is your escape hatch, not a new session. Switching sessions loses all context - including the understanding of your codebase, decisions made, and task state. /compact preserves a summary of the important bits and frees up the window, pushing you back toward SMART. The cost of a session switch (re-establishing context from scratch) almost always exceeds the marginal quality loss of the WARN zone.

Tracking It: A Claude Code Statusline

Claude Code's statusline receives a JSON payload on every update that includes context_window.remaining_percentage and context_window.context_window_size. The default approach is to show that raw remaining percentage with color coding - green when you have plenty, red when you're almost out.

That's the wrong signal. You don't need to know you have 86% of context left. You need to know you've consumed 140K tokens and are approaching the degradation threshold at 200K, and once past it, how deep into the degradation slope you are.

Here's what the zone-aware statusline looks like in practice:

  ┌─────────────────────────────────────────────────────────────────┐
  │  3bfa9040-a986-4c55  │  SMART 140K  │  ⎇ master               │
  │  Opus 4.6 (1M context)  │  my-project  │  ✓                    │
  └─────────────────────────────────────────────────────────────────┘

The SMART 140K indicator means: you're in the smart zone, with 140K tokens of headroom before the dumb zone threshold at 200K. As you consume more context, it shifts:

  SMART 140K  →  SMART 80K  →  WARN 50K  →  WARN 12K  →  DUMB +0  →  DUMB +300K  →  DUMB +600K
  (green)        (green)       (yellow)      (yellow)      (green)     (yellow)        (red)

The zone label tells you where you are, the token count tells you how deep, and the color tells you how urgent. In the SMART and WARN zones, the number counts down to the dumb threshold. In the DUMB zone, the number counts up: +NK shows how many tokens past the 200K threshold you've consumed. The color shifts from green (just crossed, low urgency) through yellow (getting stale) to red (deep degradation).

Here's the statusline script that produces this. It computes absolute tokens used from the percentage and window size, then maps to zones:

#!/usr/bin/env bash
# Claude Code status line
# Line 1: session | smart zone indicator | branch
# Line 2: model | cwd | git status (dirty/ahead/behind)

input=$(cat)

session_id=$(echo "$input" | jq -r '.session_id // empty')
model=$(echo "$input" | jq -r '.model.display_name // empty')
cwd=$(echo "$input" | jq -r '.workspace.current_dir // .cwd // empty')
remaining_pct=$(echo "$input" | jq -r '.context_window.remaining_percentage // empty')
ctx_window_size=$(echo "$input" | jq -r '.context_window.context_window_size // empty')

# Git branch + status
branch=""
git_modified=0
git_untracked=0
git_ahead=0
git_behind=0
if [ -n "$cwd" ] && { [ -d "$cwd/.git" ] || git -C "$cwd" rev-parse --git-dir >/dev/null 2>&1; }; then
  branch=$(git -C "$cwd" --no-optional-locks symbolic-ref --short HEAD 2>/dev/null)
  if [ -n "$branch" ]; then
    while IFS= read -r line; do
      case "$line" in
        '??'*) ((git_untracked++)) ;;
        *)     ((git_modified++)) ;;
      esac
    done < <(git -C "$cwd" --no-optional-locks status --porcelain 2>/dev/null)
    lr=$(git -C "$cwd" --no-optional-locks rev-list --count --left-right '@{upstream}...HEAD' 2>/dev/null)
    if [ -n "$lr" ]; then
      git_behind=$(echo "$lr" | cut -f1)
      git_ahead=$(echo "$lr" | cut -f2)
    fi
  fi
fi

# --- Session segment ---
session_part=""
if [ -n "$session_id" ]; then
  session_part=$(printf '\033[1;97m%s\033[0m' "$session_id")
fi

# --- Model segment ---
model_part=""
if [ -n "$model" ]; then
  model_part=$(printf '\033[0;36m%s\033[0m' "$model")
fi

# --- Branch segment ---
branch_part=""
if [ -n "$branch" ]; then
  branch_part=$(printf '\033[0;35m %s\033[0m' "$branch")
fi

# --- CWD segment ---
cwd_part=""
if [ -n "$cwd" ]; then
  short_cwd=$(basename "$cwd")
  cwd_part=$(printf '\033[0;33m%s\033[0m' "$short_cwd")
fi

# --- Git status segment ---
git_status_part=""
if [ -n "$branch" ]; then
  status_bits=""
  [ "$git_ahead" -gt 0 ]     && status_bits+="${git_ahead}↑"
  [ "$git_behind" -gt 0 ]    && status_bits+="${status_bits:+ }${git_behind}↓"
  [ "$git_modified" -gt 0 ]  && status_bits+="${status_bits:+ }${git_modified}M"
  [ "$git_untracked" -gt 0 ] && status_bits+="${status_bits:+ }${git_untracked}?"
  if [ -n "$status_bits" ]; then
    if [ "$git_modified" -gt 0 ] || [ "$git_untracked" -gt 0 ]; then
      git_status_part=$(printf '\033[0;31m%s\033[0m' "$status_bits")
    else
      git_status_part=$(printf '\033[0;32m%s\033[0m' "$status_bits")
    fi
  else
    git_status_part=$(printf '\033[0;32m✓\033[0m')
  fi
fi

# --- Smart Zone context segment ---
# Research: "Lost in the Middle" (Stanford 2023), "Context Rot" (Chroma 2025)
# LLM quality degrades based on ABSOLUTE token count, not fill percentage.
# A 1M-context model at 40% (400K tokens) is far more degraded than a 200K
# model at 40% (80K tokens). The attention mechanism cares about distance
# between tokens, not what fraction of the window is used.
#
# Absolute token thresholds (tunable):
#   < 100K tokens = SMART: full reasoning fidelity
#   100-200K      = WARN:  attention/quality fading
#   > 200K        = DUMB:  materially degraded, /compact recommended
smart_threshold=100000   # tokens: upper bound of full-fidelity zone
dumb_threshold=200000    # tokens: beyond here, quality is materially degraded

ctx_part=""
if [ -n "$remaining_pct" ] && [ -n "$ctx_window_size" ] \
   && [ "$ctx_window_size" -gt 0 ] 2>/dev/null; then
  remaining_int=$(printf '%.0f' "$remaining_pct")
  used_tokens=$(( ctx_window_size * (100 - remaining_int) / 100 ))

  # Tokens remaining before dumb zone (capped at 0)
  if [ "$used_tokens" -lt "$dumb_threshold" ]; then
    smart_remaining_tokens=$(( dumb_threshold - used_tokens ))
  else
    smart_remaining_tokens=0
  fi

  # Format token count for display (e.g., 85K, 142K)
  if [ "$smart_remaining_tokens" -ge 1000 ]; then
    smart_display="$(( smart_remaining_tokens / 1000 ))K"
  else
    smart_display="$smart_remaining_tokens"
  fi

  # Tokens past the dumb threshold (for DUMB zone depth display)
  dumb_depth_tokens=$(( used_tokens - dumb_threshold ))
  if [ "$dumb_depth_tokens" -lt 0 ]; then
    dumb_depth_tokens=0
  fi
  if [ "$dumb_depth_tokens" -ge 1000 ]; then
    dumb_depth_display="$(( dumb_depth_tokens / 1000 ))K"
  else
    dumb_depth_display="$dumb_depth_tokens"
  fi

  if [ "$used_tokens" -le "$smart_threshold" ]; then
    ctx_part=$(printf '\033[1;32mSMART %s\033[0m' "$smart_display")
  elif [ "$used_tokens" -lt "$dumb_threshold" ]; then
    ctx_part=$(printf '\033[1;33mWARN %s\033[0m' "$smart_display")
  else
    # DUMB sub-zones: color by urgency, not just the zone label
    if [ "$dumb_depth_tokens" -lt 100000 ]; then
      ctx_part=$(printf '\033[1;32mDUMB +%s\033[0m' "$dumb_depth_display")
    elif [ "$dumb_depth_tokens" -lt 500000 ]; then
      ctx_part=$(printf '\033[1;33mDUMB +%s\033[0m' "$dumb_depth_display")
    else
      ctx_part=$(printf '\033[1;31mDUMB +%s\033[0m' "$dumb_depth_display")
    fi
  fi
elif [ -n "$remaining_pct" ]; then
  # Fallback: no window size available, show raw remaining %
  remaining_int=$(printf '%.0f' "$remaining_pct")
  ctx_part=$(printf '\033[0;36mctx %s%%\033[0m' "$remaining_int")
fi

# --- Assemble ---
# Line 1: session | smart zone indicator | branch
# Line 2: model | cwd | git status
sep=$(printf ' \033[0;90m|\033[0m ')

line1_parts=()
[ -n "$session_part" ] && line1_parts+=("$session_part")
[ -n "$ctx_part" ]     && line1_parts+=("$ctx_part")
[ -n "$branch_part" ]  && line1_parts+=("$branch_part")

line2_parts=()
[ -n "$model_part" ]      && line2_parts+=("$model_part")
[ -n "$cwd_part" ]        && line2_parts+=("$cwd_part")
[ -n "$git_status_part" ] && line2_parts+=("$git_status_part")

join_parts() {
  local result=""
  for i in "${!parts[@]}"; do
    if [ "$i" -eq 0 ]; then
      result="${parts[$i]}"
    else
      result="${result}${sep}${parts[$i]}"
    fi
  done
  echo "$result"
}

parts=("${line1_parts[@]}")
line1=$(join_parts)

parts=("${line2_parts[@]}")
line2=$(join_parts)

if [ -n "$line2" ]; then
  printf '%b\n%b\n' "$line1" "$line2"
else
  printf '%b\n' "$line1"
fi

To use it, save the script and point your Claude Code settings at it:

{
  "statusLine": {
    "type": "command",
    "command": "~/.claude/statusline.sh",
    "padding": 0
  }
}

In the SMART and WARN zones, the number shows tokens remaining before the dumb zone threshold. In the DUMB zone, it flips: +NK shows how many tokens past the threshold you've consumed, and the color shifts from green through yellow to red based on depth. Here's how the full progression maps for a 1M-context model:

Tokens used  │  Zone   │  Display      │  Color   │  What it means
─────────────┼─────────┼───────────────┼──────────┼───────────────────────────────
0K           │  SMART  │  SMART 200K   │  green   │  Full headroom
60K          │  SMART  │  SMART 140K   │  green   │  Plenty of room
100K         │  WARN   │  WARN 100K    │  yellow  │  Quality fading, finish + compact
150K         │  WARN   │  WARN 50K     │  yellow  │  Getting close
200K         │  DUMB   │  DUMB +0      │  green   │  Just crossed. Finish task, compact
250K         │  DUMB   │  DUMB +50K    │  green   │  Low urgency — wrap up current work
300K         │  DUMB   │  DUMB +100K   │  yellow  │  Middle context fading. /compact soon
500K         │  DUMB   │  DUMB +300K   │  yellow  │  Stale. /compact when you can
700K         │  DUMB   │  DUMB +500K   │  red     │  Deep degradation. /compact NOW
950K         │  DUMB   │  DUMB +750K   │  red     │  Most early context is gone

Two variables at the top of the script control the thresholds: smart_threshold=100000 and dumb_threshold=200000. If you find your model holds up past 200K tokens, raise dumb_threshold. If you notice quality dropping earlier, lower smart_threshold. The thresholds work the same regardless of your model's window size. That's the point.

The script also includes a fallback: if context_window_size isn't available in the JSON payload, it falls back to showing the raw ctx % remaining. This handles edge cases where the API doesn't report window size.

Why Not Just Use /compact Earlier?

You could. But /compact has a cost: it summarizes your conversation, and summaries are lossy. Details, nuances, and the exact reasoning behind decisions get compressed. Running /compact too aggressively means you lose context you actually needed.

The zone model gives you a framework for when the cost of compacting is worth it:

  • In SMART zone: Don't compact. You have full fidelity. Compacting here would throw away detail for no benefit.
  • In WARN zone: Compact when you finish your current task. The model is already losing grip on early context - the summary will preserve the important parts better than the degraded attention would.
  • In DUMB zone: How urgently you compact depends on depth. At DUMB +0 to +100K, you've just crossed the threshold, so finish your task, then compact. At DUMB +500K+, most of your early context is effectively invisible and a summary is strictly better than what the degraded attention mechanism is retaining.

The Tuning Question

The 100K/200K thresholds come from the research consensus, but your mileage will vary by model and task type. Newer models may push the boundaries further out; the thresholds are intentionally conservative^[Chroma's testing found the effect kicks in around 300-400K tokens for 1M-context models. Our dumb_threshold of 200K is well below that, which gives you a comfortable margin. You could arguably push dumb_threshold higher, but the cost of being conservative (compacting slightly early) is much lower than the cost of being aggressive (working in degraded context without realizing it).].

The fact that thresholds are absolute rather than percentage-based is what makes them portable across models. The same smart_threshold=100000 and dumb_threshold=200000 work whether you're on a 200K-context model or a 1M-context model. On the smaller model, you'll hit WARN at 50% utilization. On the larger one, you'll hit it at 10%. Both are correct, because the attention mechanism's degradation depends on absolute distance, not relative fill.

If you want to calibrate: pay attention to when the model starts forgetting constraints you set earlier in the conversation, or when it contradicts a decision you already made. That's your personal WARN threshold. The statusline just makes it visible.

Takeaways

  • LLM quality degrades well before the context window fills up. The effective capacity depends on absolute token count, not percentage: a 1M model at 20% (200K tokens) is already past the degradation threshold. Marketed context size and usable context size are different numbers.
  • Degradation follows a U-shaped attention curve. The model attends well to the start and end of context but poorly to the middle. This is an architectural property of transformer attention, not a bug in any specific model.
  • Track absolute tokens, not percentages. A raw "context remaining" percentage answers the wrong question. The same percentage means wildly different things on different model sizes. You need to know how many tokens you've consumed, not what fraction of the window is left.
  • /compact is the escape hatch, not a new session. Switching sessions loses all context. Compacting preserves a summary and frees the window. The cost of a session switch almost always exceeds the marginal quality loss of the WARN zone.
  • The WARN zone is still usable. Finish your current work there, then compact. Don't panic-eject. But don't start complex new investigations there either.
  • The DUMB zone is a slope, not a cliff. The Stanford study showed a gradual U-shaped curve, not a step function. DUMB +0 (just crossed the threshold) is a completely different situation from DUMB +600K (most of your early context is invisible). Track the depth, not just the label.

Further Reading

Footnotes

  1. Liu et al., "Lost in the Middle: How Language Models Use Long Contexts," 2023. Replicated across GPT-3.5-Turbo, GPT-4, Claude 1.3, LongChat-13B, MPT-30B-Instruct, and Cohere Command. ↩

  2. The architectural root cause is RoPE (Rotary Position Embedding) long-term decay. It reduces dot-product similarity between distant token pairs. Then softmax normalization amplifies the gap by concentrating attention weight on the highest-scoring (nearest) tokens. ↩

Discuss on Twitter • View on GitHub