docs: warn that .half() corrupts CRNN recognition output on GPU + note .cuda() is required for GPU

[moduvoice]

Motivation

While benchmarking docTR's default OCR pipeline (db_resnet50 + crnn_vgg16_bn) on an NVIDIA T4 (Turing, 16GB, driver 550.163.01 / CUDA 12.4, torch 2.6.0+cu124, doctr 1.0.2a0 @ 1a1018c), I found two documentation gaps worth flagging, one of which is safety-relevant rather than just a performance nit:

1. Silent CPU fallback. The README "Quick Tour" never calls .cuda()/.to(device), so on a machine with a CUDA GPU available, ocr_predictor(...) still runs 100% on CPU with zero warning (peak_vram_mb: 0.0). The GPU section already exists in docs/using_doctr/using_models.rst, it's just not linked from the Quick Tour, so it's easy to miss. Adding the single .cuda() call measured a 17.16x speedup (2.657s → 0.155s/image) on this box.

2. .half() (fp16) silently corrupts recognition output for the default recognizer — CRITICAL. This is the important one. The officially documented half-precision path (docs/using_model_export.rst, .cuda().half()) works fine for transformer-based recognizers, but applied to the README's own default combo, crnn_vgg16_bn (an nn.LSTM-based decoder), it silently mangles the recognized text. No exception, no warning — the run completes normally and even looks faster (which it is). Example, same image, same detector, only dtype changed:

   fp32: ['Mr.', 'Anjum', 'Hameed,', 'We', 'are', 'pleased', 'inform', 'that', 'your', 'salary']
   fp16: ['Mr',  'Anjnn', 'nra,',    'Wn', 'arn', 'plansa', 'infnrn', 'tt',   'your', 'salay']
   

   This was 100% reproducible across 2 independent runs (deterministic, not a flaky numerics thing). To isolate the root cause, I fixed the detector and swapped only the recognizer: master and parseq (transformer-based recognizers, self-contained attention, no nn.LSTM) produced output identical to fp32 under the exact same .half() call. So this looks specific to the CRNN's LSTM decoder under fp16, not to half-precision inference in docTR generally. I searched existing issues (e.g. [Bug] SARNet half-precision error. #1441, a crash on a different architecture, sar_resnet31) and didn't find this silent-corruption pattern reported for crnn_vgg16_bn before.

   Since I couldn't find prior coverage of this specific case, and a silently wrong OCR result is a lot more dangerous than a slow one, I think it deserves a clearly flagged warning in the docs even though the actual fix (why the LSTM decoder is fp16-sensitive) is a code-level question I'm not proposing to touch here — see "Out of scope" below.

3. While looking into this I also checked whether the "PyTorch vs TensorFlow default backend" question (mentioned in some older docs/issues) still applies — it doesn't. As of this commit, TF has been fully removed (README tagline already says "powered by PyTorch", pyproject.toml has zero tensorflow dependencies). No doc change needed there; just confirming it's already accurate.

Changes (doc-only, no code/behavior changes)

• README.md
  • Quick Tour: added a note directly under the ocr_predictor(...) example that the predictor stays on CPU by default with no warning, how to move it to GPU, and the measured T4 speedup, linking to the existing using_models.rst GPU section.
  • Developer mode install: added a note that the unpinned torch version range can silently resolve to a CUDA build newer than the local driver supports (torch.cuda.is_available() returns False, no error), with the fix (pin the CUDA index).
• docs/source/using_doctr/using_model_export.rst
  • Added a .. warning:: right after the existing .half() example, describing the CRNN + fp16 corruption with the concrete before/after word list, and noting that master/parseq were unaffected in the same test.
• AGENTS.md (new)
  • Summarizes all of the above for future contributors/agents in one place, plus the already-safe axes I checked (no bf16 path, no SDPA/flash-attn backend selection, int8 quantization is out of scope and handled by the sister project OnnxTR).

No production code was changed — only prose additions to existing docs and a new contributor-notes file.

Out of scope (flagging as a possible follow-up issue, not fixing here)

The root cause of the CRNN + fp16 corruption (likely fp16 precision loss accumulating through the nn.LSTM gates in doctr/models/recognition/crnn/pytorch.py) is a code-level question — possibly worth an autocast-style guard around the LSTM decoder, or at minimum a runtime warning when .half() is combined with a CRNN recognizer. I'm not proposing a code fix in this PR (this account only submits doc/text changes), but I'm happy to open a separate issue with the reproduction script if that's useful — let me know and I will.

Testing / reproduction

Environment: Tesla T4 16GB, driver 550.163.01 (CUDA 12.4), Python 3.11.15, torch 2.6.0+cu124, torchvision 0.21.0+cu124, python-doctr 1.0.2a0 (editable install @ commit 1a1018c).

from doctr.io import DocumentFile
from doctr.models import ocr_predictor

# fp32 baseline (GPU, docs path)
model = ocr_predictor(det_arch="db_resnet50", reco_arch="crnn_vgg16_bn", pretrained=True).cuda()
doc = DocumentFile.from_images("sample.jpg")
result_fp32 = model(doc)

# fp16 (docs path) -- same image, same detector, only dtype changed
model_fp16 = ocr_predictor(det_arch="db_resnet50", reco_arch="crnn_vgg16_bn", pretrained=True).cuda().half()
result_fp16 = model_fp16(doc)
# -> recognized words diverge from result_fp32, deterministically, no error/warning

# root-cause isolation: swap recognizer only, keep detector + fp16
for reco in ["master", "parseq"]:
    m = ocr_predictor(det_arch="db_resnet50", reco_arch=reco, pretrained=True).cuda().half()
    # -> output identical to fp32 for both

• Ran the fp16-corruption comparison twice independently; identical corrupted output both times (deterministic).
• Ran the fp32→fp16 comparison against master and parseq recognizers (same detector/image) to isolate the corruption to the CRNN decoder specifically.
• Confirmed the .cuda() speedup and VRAM numbers via torch.cuda.max_memory_allocated() and wall-clock timing (3 runs each), CPU vs GPU-fp32 vs GPU-fp16.

Disclosure: I used an AI agent (Claude) to help investigate and benchmark this, with GPU access on a real Tesla T4, and to draft this PR. All findings above were verified by direct, reproducible execution rather than inferred.