Regex: a caller-supplied RegexMatchContext silently runs with a 32 KiB JIT stack instead of the shared 1 MiB one

[bryancall]

Summary

RegexMatchContext builds an empty PCRE2 match context, so any caller that
supplies its own context silently loses everything the shared context configures.
Today that is a 1 MiB just-in-time (JIT) stack, replaced by PCRE2's fallback of
32 KiB. plugins/regex_remap and plugins/esi are both affected.

Nobody chose 32 KiB. It is what PCRE2 uses when no stack is assigned, and it
arrived by omission.

Where

src/tsutil/Regex.cc:258 builds a blank context:

RegexMatchContext::RegexMatchContext()
{
  auto ctx = pcre2_match_context_create(nullptr);
  ...
}

src/tsutil/Regex.cc:125-130 builds the shared, thread-local one:

_match_context = pcre2_match_context_create(_general_context);
_jit_stack     = pcre2_jit_stack_create(4096, 1024 * 1024, nullptr); // 1 page min and 1MB max
pcre2_jit_stack_assign(_match_context, nullptr, _jit_stack);

Regex::exec picks between them at src/tsutil/Regex.cc:492-497: a null context
pointer gets the shared one, a supplied context gets the caller's. Every regex in
the tree takes the first path except regex_remap (regex_remap.cc:810 and
:1060) and esi (IncludeUrlValidator.h:83).

Regex::compile always JIT compiles (src/tsutil/Regex.cc:446), so the JIT stack
is the live limit, not a theoretical one.

Effect

A pattern that backtracks once per subject character exhausts 32 KiB quickly.
Measured with pcre2test from PCRE2 10.47 against the rule and the 3071 byte URL
already in our own test data (regex_remap.test.py:56 and
replay/yts-2819.replay.json):

~^/alpha/bravo/[?]((?!action=(newsfeed|calendar|contacts|notepad)).)*$~
  Minimum match limit = 9180        interpreter: MATCHES

~...~jitstack=32     Failed: error -46: JIT stack limit reached
~...~jitstack=1024   MATCHES

So a URL that should remap instead returns PCRE2_ERROR_JIT_STACKLIMIT, and
regex_remap.cc:1169 logs it and skips the rule. Same pattern, same subject, two
different answers depending on which context the caller passed.

The threshold is pattern shaped rather than length shaped. The same rule written
with a single lookahead instead of one per character needs a depth of 4 rather
than 6119, and matches fine either way.

How it got here

• Limit resources used by regex_remap to prevent crashes on stack overflow #5762 (2019) added a recursion limit to regex_remap after a production crash.
  PCRE1 recursed on the machine stack, and the estimate at the time was roughly
  500 bytes per character, so a 2000 character URL exhausted a 1 MiB thread stack.
  The limit was 2047, lowered to 1750 in regex_remap: Adjust regex recursion limit down due to crashes in testing #6819 after it still crashed in testing.
  The surviving comment said POOMA and "also dependent on actual stack size".
• regex_remap: convert from pcre to Regex #12575 (2025) converted the plugin to the shared Regex class and introduced
  RegexMatchContext so it could keep setting a limit. The new constructor did
  not pick up the JIT stack that RegexContext had been assigning since Change Regex class to use PCRE2 #11014.
• Restore regex_remap's default PCRE2 match limit #13652 removed the limit that conversion had mistranslated. The empty context
  remains, so its only present effect is to reduce the JIT stack.

The intent in 2019 was to use most of a 1 MiB thread stack without falling off
it. 32 KiB is not a descendant of that decision.

Proposed fix

Three parts, each required by the next.

1. Build RegexMatchContext by copying the shared context rather than creating a
   blank one, so a caller-supplied context cannot silently diverge, and anything
   added to the shared context later applies automatically. Callers override only
   what they intend.
2. Have the shared context resolve its JIT stack through a callback rather than
   assigning it directly. PCRE2 requires a distinct stack per thread and a copied
   context can be used on another thread, so the callback returns the calling
   thread's stack. This is the pattern pcre2jit recommends. Measured cost is
   below noise: 19.9 to 25.4 ns per match assigned directly, 20.4 to 21.7 ns
   through the callback, on a short ordinary match.
3. Assert the behavior in src/tsutil/unit_tests/test_Regex.cc rather than
   through an AuTest. The existing AuTest cannot distinguish what it is testing:
   a recursion limit error, a work limit error, a JIT stack error and no limit at
   all all render as HTTP/1.1 200 OK to curl. That is why its assertion has been
   retargeted twice, from -21 in 2019 to -47 in regex_remap: convert from pcre to Regex #12575, while the transaction it
   names now emits -46.

This needs no change to regex_remap.cc. The plugin keeps its context and the
context starts correct. It fixes esi in the same change.

Why the stack is supplied through a callback

regex_remap.cc:810 holds one RegexMatchContext per remap instance, built once at
config load and used by every ET_NET thread. That is fine today, because the shared
context it would copy from is thread_local, so each thread has its own context and
its own stack.

Copying changes that. Once a caller's context is a copy of the shared one, the copy
outlives and crosses the thread that made it, so whatever the copy carries is shared.

If the copy carried a stack pointer, every thread would backtrack into one stack:

flowchart LR
    CTX["one RegexMatchContext<br/>shared by every thread"] --> T0["ET_NET 0"]
    CTX --> T1["ET_NET 1"]
    CTX --> T2["ET_NET 2"]
    T0 --> S["one JIT stack"]
    T1 --> S
    T2 --> S
    S --> X["concurrent matches<br/>corrupt it"]
    style CTX fill:#F1EFE8,stroke:#5F5E5A,color:#2C2C2A
    style S fill:#FCEBEB,stroke:#A32D2D,color:#501313
    style X fill:#FCEBEB,stroke:#A32D2D,color:#501313

Loading

pcre2jit is explicit about this: "if you assign or pass back a non-NULL JIT stack,
this must be a different stack for each thread so that the application is thread-safe."

So the context carries a callback instead. PCRE2 invokes it at match time, on the
thread that is matching, and it returns that thread's own stack:

flowchart LR
    CTX["one RegexMatchContext<br/>shared by every thread"] --> T0["ET_NET 0"]
    CTX --> T1["ET_NET 1"]
    CTX --> T2["ET_NET 2"]
    T0 --> S0["stack for thread 0"]
    T1 --> S1["stack for thread 1"]
    T2 --> S2["stack for thread 2"]
    style CTX fill:#F1EFE8,stroke:#5F5E5A,color:#2C2C2A
    style S0 fill:#E1F5EE,stroke:#0F6E56,color:#04342C
    style S1 fill:#E1F5EE,stroke:#0F6E56,color:#04342C
    style S2 fill:#E1F5EE,stroke:#0F6E56,color:#04342C

Loading

This is the pattern the pcre2jit manual recommends for exactly this case. Verified
under ThreadSanitizer: eight threads, two thousand matches each, one shared context,
clean. Measured cost is below noise, 19.9 to 25.4 ns per match assigning a stack
directly against 20.4 to 21.7 ns through the callback, on a short ordinary match.

Behavior change this causes

regex_remap's effective JIT stack goes from 32 KiB to 1 MiB, and the 3071 byte
URL in replay/yts-2819.replay.json starts redirecting rather than falling
through to origin, so that AuTest expectation changes. The crash property from
#5762 moves to a unit test that asserts it directly.

The 2019 crash cannot recur in this form. PCRE2 has stored backtracking frames on
the heap since 10.30, and an assigned JIT stack is heap allocated, whereas the
current 32 KiB default sits on the machine stack. This reduces thread stack
pressure rather than increasing it.

Out of scope

Whether regex_remap should carry a CPU bound at all, and what it should be, is
a separate question. With PCRE2's default work limit of 10,000,000 I measured
12.4 ms for a single match against the pathological rule already in our tests,
compared to 0.002 ms under the old 1750. That deserves its own discussion rather
than being settled inside this change.

The related handling of resource errors as non matches is #13654.

Open question

AuTest 1of4 failed on a revision of #13652 that made this same change, and
passed on the seven other open pull requests at the time, so it was not
infrastructure. The author reported both regex_remap AuTests passing locally. I
have not been able to read that job's output. If the expectation holds on one host
and not another, the JIT stack threshold varies by platform and this change needs
to account for that. I will chase it before opening a pull request.

[bryancall]

The open question in this issue is answered, and the answer is that the CI failure was
never related to this change.

I said I would chase why AuTest 1of4 went red on a revision of #13652 that made this
same change, before opening a pull request. I reproduced it locally on a dev-asan
build and then baselined it. Five runs of the regex_remap AuTest for each revision,
same machine, same build settings:

revision	pass	fail
unmodified master 7ed34a3	1	4
the reverted revision dccd8f0	1	4
the fix in #13661	3	2

Every assertion in the test passes. What fails is the traffic_server exit code,
because LeakSanitizer reports a 104 byte leak from
ConfigReloadTask::start_progress_checker() at src/mgmt/config/ConfigReloadTrace.cc:394,
reached through the config reload the test performs. That is unrelated to regex
matching and it is on master right now. Filed as #13662.

Five runs each is too small to claim the fix branch is better than master; the only
claim is that the flake exists without the change.

The fix is now open as #13661. It also keeps the #5762 crash guard rather than
retiring it: the guard run moves to a subject large enough to still exhaust the 1 MB
stack, since the first -46 threshold for that rule moves from 1377 bytes to 43702
bytes once the shared stack applies.