Segfault at interpreter exit when the duckdb module outlives shutdown outside sys.modules #638

What happens?

The process segfaults at interpreter exit (exit status -11 / 139) when the duckdb module object stays alive but is no longer registered in sys.modules at shutdown. Nothing is printed by -X faulthandler, so the crash appears to happen after interpreter finalization.

A common way to get into this state is a test that uses unittest.mock.patch.dict(sys.modules): DuckDB is imported for the first time inside the block, and leaving the block restores the old snapshot, which removes duckdb/_duckdb from sys.modules. Any remaining reference, such as a sqlalchemy.ext.compiler.compiles hook registered by duckdb_engine, keeps the module alive.

Likely cause: the process-global default connection and import cache (DuckDBPyConnection::default_connection / import_cache, src/duckdb_py/pyconnection.cpp around lines 70-72 in v1.5.2) are released only by DuckDBPyConnection::Cleanup(). That function runs only when the _clean_default_connection capsule is destroyed (src/duckdb_py/duckdb_python.cpp around lines 1154-1157 in v1.5.2; the nanobind capsule on main has the same design). The interpreter clears module dicts during shutdown only for modules that are still in sys.modules, so the capsule is never destroyed while Python is alive. The C++ statics are then destroyed at exit(), after the interpreter is gone.

A cleanup that does not depend on the module dict being cleared would avoid this, e.g. one registered with Python's atexit, or Py_AtExit.

To Reproduce

import gc
import sys
from unittest import mock

with mock.patch.dict(sys.modules):
    import duckdb  # duckdb/_duckdb are removed from sys.modules on exit

def holder():
    pass

holder.module = duckdb  # any surviving reference keeps the module alive
gc.freeze()
print("exiting", duckdb.__version__, flush=True)

$ python repro.py; echo $?
exiting 1.5.5
Segmentation fault
139

With the same reference but without the patch.dict block, the process exits with status 0. It also exits with status 0 if the module is freed before shutdown.

OS:

Linux x86_64 (Debian, kernel 6.1)

DuckDB Package Version:

1.5.2 and 1.5.5

Python Version:

3.11.15

Full Name / Affiliation

N/A

Did you include all relevant configuration to reproduce the issue?

• Yes