Dictionary Changed Size During Iteration

Introduction: The Silent Killer of Your Python Loops

Imagine this: you’re carefully iterating through a dictionary, processing each key-value pair with precision. Your code looks sound, your logic seems flawless. Then, without warning, your program crashes with a cryptic error: RuntimeError: dictionary changed size during iteration. This isn't just a minor inconvenience; it's a fundamental safeguard in Python that protects your program from a subtle and dangerous form of data corruption. This error occurs when you attempt to modify a dictionary—by adding or removing keys—while you are actively looping over it using an iterator. Understanding why this happens and how to work around it is not just about fixing an error; it's about grasping a core concept of how Python manages mutable data structures and ensures iteration consistency. Mastering this principle separates programmers who write robust, predictable code from those who spend hours chasing elusive bugs.

Detailed Explanation: The Anatomy of a Dictionary Iterator

To understand the error, we must first understand what happens when you write a for loop over a dictionary. In Python, a dictionary is a mutable mapping type, implemented as a hash table. When you execute for key in my_dict:, Python doesn't simply create a static list of all keys and walk through it. Instead, it creates a special iterator object for that specific dictionary. This iterator is a lightweight, stateful object that knows exactly where it is in the dictionary's internal structure and what it expects to see next.

The iterator's job is to traverse the dictionary's underlying array of hash table entries in a specific order. It maintains an internal index or pointer. Each time the loop asks for the next item, the iterator consults the dictionary's current state, finds the next occupied slot, and returns its key. The critical contract is this: the iterator assumes the dictionary's structure—its size and the layout of its hash table—remains constant for the duration of the iteration. If this assumption is violated, the iterator's internal pointer can become misaligned, potentially skipping items, visiting the same item twice, or accessing invalid memory, leading to undefined behavior. To prevent this chaos, Python's C implementation performs an explicit check. If the dictionary's size (the count of items) changes between iterator steps, it immediately raises the RuntimeError. This is a fail-fast mechanism, alerting you to a logic error before it can cause silent, incorrect results.

Step-by-Step Breakdown: The Moment of Failure

Let's walk through the precise sequence of events that triggers the error.

1. Iterator Creation: You write for key in my_dict:. Python calls my_dict.__iter__() to create a dictionary iterator object. At this moment, the iterator records the dictionary's current "version" or "size" (in CPython, it stores the dictionary's ma_version_tag). It also initializes its internal index to the start of the hash table array.
2. First Iteration: The loop requests the first item. The iterator starts at index 0, scans the hash table until it finds an occupied slot, and yields that key. The dictionary's size is still unchanged.
3. The Fateful Modification: Inside the loop body, your code executes an operation that changes the dictionary's size. This could be:
   • my_dict['new_key'] = 'new_value' (Addition)
   • del my_dict['existing_key'] (Deletion)
   • my_dict.pop('key') (Deletion)
   • my_dict.clear() (Massive Deletion)
   • my_dict.update(...) (Potential Additions/Deletions)
4. The Check: Before the loop proceeds to the next iteration and asks the iterator for the next key, Python's iterator machinery compares the dictionary's current size with the size it recorded at creation. They differ. The contract is broken.
5. The Error: The interpreter immediately raises RuntimeError: dictionary changed size during iteration. The loop terminates abruptly. Your program's flow is interrupted, and any cleanup code after the loop (in a finally block, for instance) may not run as expected.

It's crucial to note that the error is raised on the next iteration attempt, not necessarily at the exact line where the modification occurs. The modification happens, but the guardrail only triggers when the iterator tries to continue its journey.

Real Examples: From Pitfall to Solution

Example 1: The Classic Pitfall

data = {'a': 1, 'b': 2, 'c': 3}
for key in data:
    print(key)
    if key == 'b':
        data['d'] = 4  # Modifying during iteration!

Output: a, b, then RuntimeError. Why: After processing 'b', we add 'd'. When the loop tries to get the next key, the iterator detects the dictionary grew from 3 to 4 items and fails.

Example 2: The Deletion Trap

data = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
for key in list(data):  # Safe! We iterate over a list copy of keys.
    print(key)
    if key == 'b':
        del data['c']  # Safe! We're not iterating over the live dict.

Output: a, b, c, d (Note: 'c' is printed even though we delete it after printing 'b' but before the loop would have reached 'c' in the original order. This is because we're iterating over the snapshot list(data)). Why This Works: list(data) creates a new, independent list containing all keys at that moment. The for loop iterates over this static list. Modifying the original data dictionary during this process is perfectly safe because the iterator is