The with Statement Deconstructed: A Practical Deep Dive into Python's contextlib

Almost every Python programmer writes with open(...) as f: on their first day, then never thinks about what the with statement actually does. That is a shame, because context managers are one of the language's most elegant features for handling setup and teardown: opening and closing files, acquiring and releasing locks, starting and committing (or rolling back) transactions, or simply making sure a resource is cleaned up no matter what goes wrong. Once you understand the protocol underneath with, you can replace piles of fragile try/finally code with a few readable lines. This guide walks through how context managers work, how to build your own, and how the contextlib module gives you a toolbox of ready-made helpers.

The problem context managers solve

Consider code that needs a resource cleaned up even when an exception is raised. Without a context manager, you write this:

f = open("data.txt", "w", encoding="utf-8")
try:
    f.write("some important data")
    risky_operation()
finally:
    f.close()

The try/finally is easy to forget and grows noisy when you juggle several resources. The with statement captures exactly this pattern — "do something on entry, guarantee something on exit" — and hides the boilerplate:

with open("data.txt", "w", encoding="utf-8") as f:
    f.write("some important data")
    risky_operation()
# f is closed here, exception or not

The file is closed whether the block finishes normally, returns early, or raises. That guarantee is the whole point.

The context manager protocol

Any object can be a context manager if it implements two dunder methods: __enter__ and __exit__. When Python hits a with block it calls __enter__ (whose return value is bound to the as variable), runs the body, then calls __exit__ on the way out. Here is a small timer that measures how long its block takes:

import time

class Timer:
    def __enter__(self):
        self.start = time.perf_counter()
        return self          # bound to the "as" name

    def __exit__(self, exc_type, exc_val, exc_tb):
        self.elapsed = time.perf_counter() - self.start
        print(f"Elapsed: {self.elapsed:.6f}s")
        return False         # do not suppress exceptions

with Timer() as t:
    sum(range(1_000_000))

The three arguments to __exit__ describe any exception that escaped the block: its type, value, and traceback. If the block finished cleanly they are all None. The return value of __exit__ matters: return a falsy value (like False or None) and any exception propagates normally; return True and Python swallows the exception. Use that power sparingly:

class IgnoreValueError:
    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        return exc_type is ValueError   # suppress only ValueError

with IgnoreValueError():
    raise ValueError("this gets swallowed")

print("execution continues here")

A cleaner way: the @contextmanager decorator

Writing a whole class for a simple setup/teardown pair is heavy. The contextlib.contextmanager decorator lets you write a context manager as a generator instead. Everything before the yield is the "enter" phase; everything after is the "exit" phase. Whatever you yield becomes the as value.

from contextlib import contextmanager

@contextmanager
def tag(name):
    print(f"<{name}>")
    try:
        yield
    finally:
        print(f"</{name}>")

with tag("p"):
    print("content")
# prints: <p> / content / </p>

The try/finally around the yield is important: it ensures the teardown runs even if the with body raises. This pattern shines for managing a resource with real cleanup — for example a temporary file that must always be deleted:

import os, tempfile
from contextlib import contextmanager

@contextmanager
def temp_file(text):
    fd, path = tempfile.mkstemp()
    with os.fdopen(fd, "w") as fh:
        fh.write(text)
    try:
        yield path            # hand the path to the caller
    finally:
        os.remove(path)       # always clean up

with temp_file("scratch data") as path:
    print("working with", path)
# file is gone once the block exits

Batteries in contextlib

The contextlib module ships several ready-made managers that remove common boilerplate.

suppress: ignore specific exceptions

Instead of an empty except block, say exactly what you mean:

import os
from contextlib import suppress

with suppress(FileNotFoundError):
    os.remove("maybe_missing.tmp")   # no error if it isn't there

redirect_stdout: capture printed output

Useful for testing code that prints, or for grabbing output from a library you cannot modify:

import io
from contextlib import redirect_stdout

buffer = io.StringIO()
with redirect_stdout(buffer):
    print("this goes into the buffer")

print("captured:", repr(buffer.getvalue()))

There is a matching redirect_stderr for the error stream.

closing: adapt objects that only have close()

Some objects offer a close() method but do not implement the context-manager protocol. closing wraps them so with still works:

from contextlib import closing
from urllib.request import urlopen

with closing(urlopen("https://example.com")) as page:
    data = page.read()
# page.close() is called automatically

Managing many resources with ExitStack

Sometimes you do not know how many resources you need until runtime — say, opening a list of files whose length varies. Nesting with statements does not work when the count is dynamic. ExitStack solves this by letting you register context managers and cleanup callbacks programmatically; everything is unwound in LIFO order when the stack closes.

from contextlib import ExitStack

filenames = ["a.txt", "b.txt", "c.txt"]

with ExitStack() as stack:
    files = [
        stack.enter_context(open(name, "w", encoding="utf-8"))
        for name in filenames
    ]
    for f in files:
        f.write("hello\n")
# every file is closed here, in reverse order

Beyond enter_context, you can register arbitrary cleanup functions with callback. They too run in reverse registration order:

from contextlib import ExitStack

with ExitStack() as stack:
    stack.callback(print, "cleanup 1")
    stack.callback(print, "cleanup 2")
    print("body runs")
# prints: body runs / cleanup 2 / cleanup 1

Two more handy tools

ContextDecorator: use a manager as a decorator

Subclass ContextDecorator (or use @contextmanager, which already supports this) and your context manager can wrap a whole function without an explicit with:

from contextlib import ContextDecorator

class banner(ContextDecorator):
    def __enter__(self):
        print("== start ==")
        return self
    def __exit__(self, *exc):
        print("== end ==")
        return False

@banner()
def do_work():
    print("working")

do_work()   # prints start / working / end

nullcontext: an optional context manager

When a resource is optional, nullcontext lets you keep a single code path instead of branching. It does nothing on enter and exit:

from contextlib import nullcontext

def process(data, lock=None):
    cm = lock if lock is not None else nullcontext()
    with cm:               # real lock, or a harmless no-op
        return sum(data)

Common pitfalls

A few traps catch people repeatedly. First, in a @contextmanager generator, always wrap the yield in try/finally — otherwise an exception in the body skips your cleanup. Second, remember that returning True from __exit__ (or catching the exception inside a @contextmanager) silently suppresses errors; only do it deliberately. Third, a @contextmanager generator is single-use — you cannot reuse the same instance across two with blocks; call the factory again each time. Finally, do not reach for ExitStack when a plain nested with (or a comma-separated with open(a) as x, open(b) as y:) is clearer; save it for the genuinely dynamic cases.

Wrap-up and next steps

Context managers turn "make sure this always gets cleaned up" from an error-prone habit into a language guarantee. Start by recognizing the __enter__/__exit__ protocol, reach for @contextmanager for quick custom managers, and lean on contextlib's built-ins — suppress, redirect_stdout, closing, ExitStack, and nullcontext — to delete boilerplate. As a next step, try refactoring a function in your own code that uses try/finally into a context manager, and look at how libraries you already use (database drivers, threading.Lock, tempfile) expose their resources through this same protocol. Once it clicks, you will start seeing setup/teardown pairs everywhere — and reaching for with to tame them.