[BUG] Stop the Elasticsearch async ForceFlush reporting success without waiting

[thc1006]

Fixes #4336.

ForceFlush waited for options_.response_timeout_ rather than the timeout the caller gave it, and the branch that ran out of time returned true whatever had happened:

while (timeout_steady > std::chrono::steady_clock::duration::zero())
{
  ...
  timeout_steady -= std::chrono::steady_clock::now() - start;
}
return timeout_steady > std::chrono::steady_clock::duration::zero();

The loop condition has already established that the value is positive, so every flush that ran out of time reported success.

Two more ways it answered without having waited. It compared monotonic totals with no session identity, so a completion from a session started after the call could satisfy a waiter for one started before it. And a batch already inside Export() was not covered at all, because the session was registered after the request had been built.

One steady_clock deadline, taken at function entry so waiting for the mutex spends the caller's budget, and a wait on a predicate over the set of running session ids against a watermark. wait_until returns the predicate, so the return value answers the caller's question rather than being a leftover duration. Registration moves to the top of Export(). The serialising lock is gone rather than made timed: each call snapshots what it waits for and publishes nothing, so two callers were already safe side by side. Measured at 2921 ms for a ForceFlush(20ms) queued behind a 3 second one.

This is stacked on #4502. That pull request carries the exactly-once completion this accounting depends on, and closes #4338. GitHub cannot take a fork branch as a base, so until #4502 lands the diff here shows both; this half is 861 of the lines. Review #4502 first.

Twenty cases: sixteen for the flush, and four that check exactly-once through the flush rather than through the completion log line. Reverting each defect turns cases red: the deadline change turns three red, the counter comparison turns the two substitution cases red, and moving registration back below the request build turns AnExportAlreadyUnderWayIsSomethingToWaitFor red.

Not fixed here: Shutdown(timeout) still ignores its timeout, and true means every export the call snapshotted reported a terminal outcome, not that its batch reached Elasticsearch, which is #3075.

For significant contributions please make sure you have completed the following items:

• CHANGELOG.md updated for non-trivial changes
• Unit tests have been added
• Changes in public API reviewed

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.80%. Comparing base (58ed80d) to head (c4033ed).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4337      +/-   ##
==========================================
+ Coverage   83.24%   83.80%   +0.57%     
==========================================
  Files         521      521              
  Lines       20384    20400      +16     
==========================================
+ Hits        16967    17095     +128     
+ Misses       3417     3305     -112     

Files with missing lines	Coverage Δ
...y/exporters/elasticsearch/es_log_record_exporter.h	100.00% <ø> (ø)
...orters/elasticsearch/src/es_log_record_exporter.cc	96.72% <100.00%> (+84.50%)	⬆️

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[thc1006]

The one red is Bazel asan config, and it is not from this diff. BasicCurlHttpTests.ElegantQuitQuick in //ext/test/http:curl_http_test missed its 20 ms wall clock budget at curl_http_test.cc:940. That is the flake #4342 described and #4488 fixes; #4342 is closed, the flake is not.

This branch changes five files, all under exporters/elasticsearch/, so rather than argue it from the diff:

$ bazel query "somepath(//ext/test/http:curl_http_test, //exporters/elasticsearch/...)"
INFO: Empty results

The same job is green on main at 70fdb766, the commit this is rebased onto.

I cannot re-run a job here, so it will clear on the next push or on a maintainer re-run.