Skip to content

[BUG FIX]: hf_hub_download crashes when stderr lacks a real file descriptor#4065

Open
tobocop2 wants to merge 3 commits intohuggingface:mainfrom
tobocop2:fix/tqdm-bad-fd-safety
Open

[BUG FIX]: hf_hub_download crashes when stderr lacks a real file descriptor#4065
tobocop2 wants to merge 3 commits intohuggingface:mainfrom
tobocop2:fix/tqdm-bad-fd-safety

Conversation

@tobocop2
Copy link
Copy Markdown

@tobocop2 tobocop2 commented Apr 7, 2026

Problem

hf_hub_download crashes with ValueError: bad value(s) in fds_to_keep in any environment where sys.stderr is not backed by a real file descriptor.

The download itself is fine. Only tqdm's progress bar initialization fails, but because _create_progress_bar has no safety net, it takes down the entire download.

Affected environments

Any runtime where sys.stderr is not backed by a real file descriptor:

  • Textual TUI apps
  • Jupyter notebooks
  • pytest capture
  • WSGI/ASGI web servers (gunicorn, uvicorn)
  • Sandboxed/containerized runtimes
  • IDE embedded terminals

Root cause

_create_progress_bar calls cls(disable=disable, name=name, **kwargs) with no exception handling. When stderr's fileno() returns -1, tqdm's __init__ tries to create a multiprocessing lock, which spawns a resource tracker subprocess via fork_exec(), which rejects the invalid fd.

Crash Demo

Crash demo

Fix

Add a _SafeTqdm subclass that uses a threading.RLock instead of a multiprocessing lock, and wrap both cls() calls in _create_progress_bar with try/except (OSError, ValueError) to fall back to _SafeTqdm.

Fixes #4066


Note

Low Risk
Change is limited to progress-bar creation error handling and test coverage; it only activates on tqdm initialization failures and otherwise preserves existing behavior.

Overview
Prevents downloads from crashing when tqdm progress bar initialization fails (notably when sys.stderr lacks a valid file descriptor) by wrapping progress-bar construction in try/except and falling back to a disabled _SafeTqdm that uses a thread lock.

Adds a user warning for the default HF tqdm path when progress reporting is unavailable, and extends tests to cover the bad-fileno scenario and verify that both custom and HF tqdm_class failures degrade to disable=True (without emitting pytest unraisable destructor warnings).

Reviewed by Cursor Bugbot for commit 2e89815. Bugbot is set up for automated code reviews on this repo. Configure here.

@tobocop2 tobocop2 force-pushed the fix/tqdm-bad-fd-safety branch from f08d2e6 to 6eff9fb Compare April 7, 2026 21:52
@tobocop2 tobocop2 changed the title fix: handle tqdm crash when stderr lacks a real file descriptor fix: handle hf_hub_download crash when stderr lacks a real file descriptor Apr 7, 2026
@tobocop2 tobocop2 force-pushed the fix/tqdm-bad-fd-safety branch from 6f4b2a4 to 43399d3 Compare April 8, 2026 03:52
@tobocop2 tobocop2 changed the title fix: handle hf_hub_download crash when stderr lacks a real file descriptor fix: hf_hub_download crashes when stderr lacks a real file descriptor Apr 8, 2026
@tobocop2 tobocop2 changed the title fix: hf_hub_download crashes when stderr lacks a real file descriptor [BUG FIX]: hf_hub_download crashes when stderr lacks a real file descriptor Apr 8, 2026
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 59aa035. Configure here.

Comment thread src/huggingface_hub/utils/tqdm.py Outdated
@tobocop2 tobocop2 force-pushed the fix/tqdm-bad-fd-safety branch 2 times, most recently from 98872dc to cf1615c Compare April 13, 2026 17:11
Add _SafeTqdm fallback with thread lock for environments where
stderr.fileno() is invalid (Textual, Jupyter, pytest, WSGI, etc.).
Wrap progress bar construction in try/except to degrade gracefully
instead of crashing the download.
@tobocop2 tobocop2 force-pushed the fix/tqdm-bad-fd-safety branch from cf1615c to 037f4ad Compare April 13, 2026 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: hf_hub_download crashes with ValueError when stderr lacks a real file descriptor

1 participant