Python Generators

Overview

Generators produce values one at a time using yield, pausing between items. They are iterators without storing the full sequence in memory—ideal for large files, infinite streams, and pipeline processing.

Syntax / Usage

# Generator function
def countdown(n):
    while n > 0:
        yield n
        n -= 1

for i in countdown(3):
    print(i)  # 3, 2, 1

# Generator expression (like list comp but lazy)
squares = (x ** 2 for x in range(1_000_000))

# Send values into a generator
def echo():
    while True:
        received = yield
        print(received)

gen = echo()
next(gen)       # prime the generator
gen.send("hi")  # prints hi

# yield from delegates to sub-iterator
def chain_lists(a, b):
    yield from a
    yield from b

Examples

Read a large log file line by line without loading it all:

def read_lines(path):
    with open(path) as f:
        for line in f:
            yield line.strip()

for line in read_lines("huge.log"):
    if "ERROR" in line:
        print(line)

Pipeline: filter then transform:

def evens(numbers):
    for n in numbers:
        if n % 2 == 0:
            yield n

def double(numbers):
    for n in numbers:
        yield n * 2

result = list(double(evens(range(10))))
# [0, 4, 8, 12, 16]

Common Mistakes

• Exhausting a generator—iterators are single-use; create a new one to replay
• Using a generator when you need random access or len()—use a list
• Forgetting to prime generators that use .send()
• Confusing generator expressions () with tuples (x,) when no comprehension syntax is used

See Also

python-list-comprehension python-loops python-decorators