Desk Clarity

Everyday document & browser help

Python Popen Timeout: Separate Running Status, Output Collection and Child Cleanup

By ON Lab · Sources checked 8 November 2026

Popen.poll() checks whether a child has exited; communicate() collects its piped output and waits for completion. A timeout on communicate() stops that wait by raising TimeoutExpired, but it does not itself kill the child. Decide who owns cancellation before treating a timeout as a completed job.

Separate status, output and success

The original reader launched programs in the background and found that communicate() blocked monitoring. Their condition also confused exit status: Python uses None for a child that is still running, while a completed exit status of 0 usually means success. A truthiness test skips 0, so it cannot distinguish success from running correctly.

poll() is useful for a status snapshot. It does not return captured stdout or stderr, and it does not turn a pipe into a nonblocking output stream. If your program owns several long-lived children, a loop that waits on one child's output may still prevent other work. This guide limits its runnable example to a single direct child with tiny bounded output.

Observe a timeout without leaving the child running

Run this self-contained example in a Python terminal. It starts only a short Python child created by this script, not an existing user process. The child prints one line and sleeps; the parent records whether it is still running after a short wait:

import subprocess
import sys

child_code = "import time; print('started', flush=True); time.sleep(5)"
process = subprocess.Popen(
    [sys.executable, "-c", child_code],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    text=True,
)
try:
    try:
        output, errors = process.communicate(timeout=0.2)
        print("completed", process.returncode, repr(output))
    except subprocess.TimeoutExpired:
        print("still running", process.poll() is None)
        process.kill()
        output, errors = process.communicate(timeout=2)
        print("collected", repr(output), repr(errors))
        print("exit status", process.returncode)
finally:
    if process.poll() is None:
        process.kill()
        process.communicate(timeout=2)

On our Python 3.13.18 macOS run, the timeout branch printed still running True, collected 'started\n' and empty stderr, then reported exit status -9. The negative status reflects this POSIX child being killed by a signal. Do not expect that exact numeric status on Windows.

The example uses a five-second sleep to make a short timeout likely in a normal run. Scheduling and process creation can vary, so it also handles the normal-completion branch. The two-second cleanup bound is a demonstration choice, not a universal production deadline. If cleanup itself times out, the exception remains visible instead of reporting the job as finished.

Retry communication after cancellation, not a new launch

After a communicate timeout, the direct Popen object still owns the same child and its pipes. Killing it and calling communicate again finishes collecting the bounded output and updates its returncode. Do not automatically start another job while the original may still be running: overlapping work can create duplicate side effects.

A timeout alone also does not mean the child failed. It means that this wait did not finish within its timeout. A child could be slow, blocked on a dependency, producing excessive output, or intentionally long-running. Choose a cancellation or continued-wait policy that matches your application before retrying.

The cleanup here follows the official documentation's direct-child pattern. It is not a graceful shutdown protocol and does not establish that descendants have stopped. Children or external tools can create their own subprocesses and keep output handles open. Process-tree ownership, graceful termination and platform-specific cancellation require a separate design.

Check a completed exit status explicitly

import subprocess
import sys

process = subprocess.Popen(
    [sys.executable, "-c", "print('finished')"],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    text=True,
)
output, errors = process.communicate(timeout=2)
status = process.poll()
if status is None:
    print("still running")
elif status == 0:
    print("success", repr(output))
else:
    print("failed", status, repr(errors))

Our normal fixture printed success 'finished\n'. The explicit None and 0 comparisons preserve the difference between running and successful completion. A nonzero code generally indicates failure by the program's convention, but you still need its documented meaning and diagnostic output before deciding how to recover.

Know when this collection pattern does not fit

communicate buffers the collected data in memory. Do not apply this small-output example to an unlimited log stream. If output is not needed, avoid creating unnecessary pipes; if it is needed continuously, choose a streaming strategy appropriate to your platform and application. Merely polling the child without draining its pipes can allow a full pipe buffer to block the child.

subprocess.run with a timeout offers a simpler contract for a single operation: its documented timeout behavior kills and waits for its child before re-raising TimeoutExpired. That differs from using Popen.communicate directly. It still does not promise a strict end-to-end wall-clock maximum, because process creation itself may not be interruptible on some platform APIs.

Do not replace a shell command blindly just to copy this example. The example passes an argument list to the current interpreter without a shell. Argument interpretation is a separate boundary covered in the POSIX shell argument guide. Neither this article nor the original discussion proves how your external program handles cancellation.

Record enough to make the next decision

For a controlled failure, retain whether the timeout occurred, whether the direct child remained running, the eventual returncode, and the permitted diagnostic output. Avoid logging passwords or private command inputs. Confirm the prior attempt's disposition before considering a retry. A captured receipt or exception is not evidence that downstream work completed successfully.

Method: We ran synthetic timeout and normal-exit cases using Python 3.13.18 on macOS, then executed both complete published examples separately. The fixtures launched only their own direct child, produced a tiny line of output and collected it after cleanup. We did not test Windows, descendants, external services, multiple-worker scheduling or unbounded logs. The historical question establishes the reader problem; the current Python documentation and local fixture establish the narrow behavior explained here.

Sources