Looping Without the Loops: A Practical Deep Dive into Python's itertools

Most Python developers reach for a for loop, a list comprehension, and maybe a temporary list or two whenever they need to transform a sequence. That works — until the data gets large, the loops get nested, and you find yourself building intermediate lists just to throw them away. The itertools module offers a different mental model: instead of materializing data, you compose small, lazy iterator building blocks into pipelines that process one element at a time.

Because everything in itertools is an iterator, values are produced on demand and memory usage stays flat no matter how big the input is. It's a pure C implementation, it's in the standard library (no install needed), and once the vocabulary clicks, a surprising amount of "loop plumbing" collapses into a single expressive line. Let's walk through the pieces that matter most in day-to-day code.

The infinite iterators: count, cycle, and repeat

Three functions produce endless streams. On their own they'd loop forever, so you always pair them with something that stops — zip, islice, or a takewhile.

from itertools import count, cycle, repeat

# count(start, step) — an endless counter, like a lazy range with no end
c = count(10, 2)
print([next(c) for _ in range(4)])      # [10, 12, 14, 16]

# cycle — repeat a sequence forever (great for round-robin assignment)
colors = cycle(["red", "green", "blue"])
print([next(colors) for _ in range(5)]) # ['red', 'green', 'blue', 'red', 'green']

# repeat — the same value, optionally a fixed number of times
print(list(map(pow, range(5), repeat(2))))  # [0, 1, 4, 9, 16]

The count function is especially handy for pairing indices with values in a lazy pipeline, and repeat shines when a function like map needs a constant second argument.

Combining and filtering streams

The real payoff comes from stitching iterables together. chain concatenates them lazily, compress filters by a parallel mask, and takewhile/dropwhile slice a stream based on a condition rather than an index.

from itertools import chain, compress, takewhile, dropwhile, islice, count

# chain flattens multiple iterables into one stream — no intermediate list
print(list(chain([1, 2], [3, 4], [5])))          # [1, 2, 3, 4, 5]
print(list(chain.from_iterable([[1, 2], [3, 4]]))) # [1, 2, 3, 4]

# compress keeps items where the selector is truthy
print(list(compress("ABCDEF", [1, 0, 1, 0, 1, 1])))  # ['A', 'C', 'E', 'F']

# takewhile stops at the first False; dropwhile skips until the first False
nums = [1, 4, 6, 2, 1]
print(list(takewhile(lambda x: x < 5, nums)))   # [1, 4]
print(list(dropwhile(lambda x: x < 5, nums)))   # [6, 2, 1]

# islice is "slicing for iterators" — start, stop, step, but no negative indices
print(list(islice(count(), 2, 10, 2)))          # [2, 4, 6, 8]

islice deserves special mention: you can't write my_generator[2:10], but islice(my_generator, 2, 10) does exactly that without consuming the whole thing. It's the standard way to take the "first N" items from an infinite or expensive stream.

Running totals with accumulate

accumulate returns running results of a binary function. By default it sums, but you can pass any two-argument function — max, operator.mul, or your own — and an initial value.

from itertools import accumulate
import operator

print(list(accumulate([1, 2, 3, 4, 5])))              # [1, 3, 6, 10, 15]
print(list(accumulate([3, 1, 4, 1, 5, 9], max)))       # [3, 3, 4, 4, 5, 9]  running maximum
print(list(accumulate([1, 2, 3], operator.mul)))       # [1, 2, 6]           running product
print(list(accumulate([1, 2, 3], operator.add, initial=100)))  # [100, 101, 103, 106]

Running maximums, cumulative sums for a chart, or compound growth all become one line — and because it's lazy, you can feed the result straight into the next stage of a pipeline.

Grouping consecutive items with groupby

groupby collapses adjacent items that share a key into groups. The word "adjacent" is the whole pitfall: if your data isn't sorted by the grouping key, you'll get fragmented groups. Sort first, then group.

from itertools import groupby
import operator

data = [
    ("fruit", "apple"), ("fruit", "banana"),
    ("veg", "carrot"), ("veg", "pea"),
    ("fruit", "cherry"),
]

# Sort by the same key you group by — otherwise 'fruit' appears twice
data.sort(key=operator.itemgetter(0))

for key, group in groupby(data, key=operator.itemgetter(0)):
    print(key, [item[1] for item in group])
# fruit ['apple', 'banana', 'cherry']
# veg ['carrot', 'pea']

Note the second gotcha: the group iterator is shared and consumed as you advance the outer loop. If you need the members later, materialize each group into a list (as above) before moving on.

Combinatorics: product, permutations, and combinations

Nested loops for "every pairing" are a classic readability trap. product replaces nested for loops, while permutations and combinations generate ordered and unordered selections.

from itertools import product, permutations, combinations, combinations_with_replacement

# product is the Cartesian product — replaces nested for-loops
print(list(product([0, 1], repeat=2)))     # [(0,0), (0,1), (1,0), (1,1)]

# permutations: order matters (r-length ordered tuples)
print(list(permutations("ABC", 2)))         # [('A','B'), ('A','C'), ('B','A'), ...]

# combinations: order does not matter
print(list(combinations("ABCD", 2)))        # [('A','B'), ('A','C'), ('A','D'), ('B','C'), ...]

# with replacement allows repeats
print(list(combinations_with_replacement("AB", 2)))  # [('A','A'), ('A','B'), ('B','B')]

These are lazy generators, so iterating over product(range(1000), repeat=3) won't try to build a billion-element list up front — though you should still be mindful of how many you actually consume.

Useful helpers: starmap, zip_longest, pairwise, and tee

from itertools import starmap, zip_longest, pairwise, tee

# starmap is map() when your arguments are already packed in tuples
print(list(starmap(pow, [(2, 3), (3, 2), (10, 2)])))   # [8, 9, 100]

# zip_longest pads the shorter iterable instead of stopping early
print(list(zip_longest("ABC", "xy", fillvalue="-")))    # [('A','x'), ('B','y'), ('C','-')]

# pairwise (Python 3.10+) yields overlapping consecutive pairs
print(list(pairwise([1, 2, 3, 4])))                     # [(1,2), (2,3), (3,4)]

# tee splits one iterator into several independent ones
source = iter([1, 2, 3, 4])
a, b = tee(source, 2)
print(list(a), list(b))                                 # [1,2,3,4] [1,2,3,4]

A word of caution on tee: once you tee an iterator, stop using the original, and be aware that if one branch races far ahead of the other, tee buffers everything in between. For two branches consumed roughly in step it's cheap; for wildly divergent consumption, a list may be simpler.

Two recipes worth memorizing

The official docs include a "recipes" section — small compositions that solve common problems. Two come up constantly. A sliding window over a sequence:

from itertools import islice

def sliding_window(iterable, n):
    it = iter(iterable)
    window = tuple(islice(it, n))
    if len(window) == n:
        yield window
    for x in it:
        window = window[1:] + (x,)
        yield window

print(list(sliding_window([1, 2, 3, 4, 5], 3)))
# [(1, 2, 3), (2, 3, 4), (3, 4, 5)]

And batching a stream into fixed-size chunks. On Python 3.12+ this is built in as itertools.batched; on older versions the islice pattern does the job:

from itertools import islice

def batched(iterable, n):
    it = iter(iterable)
    while batch := tuple(islice(it, n)):
        yield batch

print(list(batched("ABCDEFG", 3)))
# [('A', 'B', 'C'), ('D', 'E', 'F'), ('G',)]

# Python 3.12+: from itertools import batched  # same behavior, in C

Common pitfalls

A few sharp edges catch newcomers. Iterators are single-use: once you exhaust an itertools object it's empty, so you can't iterate it twice — wrap it in list() if you need to. The infinite iterators (count, cycle, repeat without a count) will hang forever if you call list() on them; always bound them with islice, zip, or takewhile. And remember that groupby only groups runs of adjacent equal keys, so sort by the key first unless you specifically want run-length behavior.

Wrap-up and next steps

The value of itertools isn't any single function — it's the composability. Once you think in terms of lazy streams, you can pipe chain into islice into groupby and process gigabytes with constant memory and no nested loops. Start small: replace one place where you build a throwaway list with chain.from_iterable, or swap a manual running total for accumulate. Then read the "Itertools Recipes" section at the bottom of the official documentation — it's a masterclass in composing these primitives, and the more-itertools package on PyPI packages many of those recipes for you. Your loops will get shorter, and your memory profile will thank you.