feat(data-pipeline): decouple OTLP trace timeout from agent connection timeout#2242
feat(data-pipeline): decouple OTLP trace timeout from agent connection timeout#2242bm1549 wants to merge 1 commit into
Conversation
…n timeout `set_connection_timeout` drives both the OTLP export request timeout and the agent `Endpoint.timeout_ms` used for agent info, stats, Remote Config, and telemetry. A host tracer that maps `OTEL_EXPORTER_OTLP_TRACES_TIMEOUT` onto `set_connection_timeout` therefore silently reshapes agent-side request timeouts: a small OTLP timeout shortens them (breaking RC/stats/telemetry), and the OTLP default (10s) even loosens the agent's 3s default. Add a dedicated `set_otlp_timeout` builder setter (and the FFI equivalent `ddog_trace_exporter_config_set_otlp_timeout`) that feeds only the OTLP trace config timeout. The agent `Endpoint.timeout_ms` stays driven solely by `connection_timeout`. When `otlp_timeout` is unset the OTLP trace timeout falls back to `connection_timeout` and then the default, so existing callers are unaffected. The OTLP metrics-export timeout stays driven by `connection_timeout` only, so a traces-scoped `otlp_timeout` never shortens OTLP metrics/stats requests. Timeout resolution is centralized in `resolve_otlp_timeouts` and unit-tested across all otlp/connection combinations, alongside a builder test asserting the agent endpoint keeps its own timeout and an FFI setter test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
📚 Documentation Check Results📦
|
🔒 Cargo Deny Results📦
|
BenchmarksComparisonBenchmark execution time: 2026-07-16 01:56:31 Comparing candidate commit 4880dd4 in PR branch Found 0 performance improvements and 2 performance regressions! Performance is the same for 22 metrics, 0 unstable metrics.
|
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4880dd4 | 1784166212 | brian.marks/otlp-timeout-decouple |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| trace_buffer/1_senders/10us_delay | execution_time | 56.289ms | 56.552ms ± 0.133ms | 56.531ms ± 0.075ms | 56.616ms | 56.810ms | 56.962ms | 57.065ms | 0.95% | 1.015 | 1.565 | 0.23% | 0.009ms | 1 | 200 |
| trace_buffer/1_senders/10us_delay | throughput | 15771.514op/s | 15914.512op/s ± 37.230op/s | 15920.602op/s ± 21.058op/s | 15937.783op/s | 15961.231op/s | 15979.615op/s | 15988.943op/s | 0.43% | -0.997 | 1.515 | 0.23% | 2.633op/s | 1 | 200 |
| trace_buffer/1_senders/1us_delay | execution_time | 50.011ms | 50.077ms ± 0.027ms | 50.074ms ± 0.011ms | 50.086ms | 50.136ms | 50.159ms | 50.207ms | 0.27% | 1.249 | 3.250 | 0.05% | 0.002ms | 1 | 200 |
| trace_buffer/1_senders/1us_delay | throughput | 17925.634op/s | 17972.234op/s ± 9.777op/s | 17973.393op/s ± 4.030op/s | 17976.991op/s | 17986.376op/s | 17990.341op/s | 17996.053op/s | 0.13% | -1.243 | 3.229 | 0.05% | 0.691op/s | 1 | 200 |
| trace_buffer/1_senders/no_delay | execution_time | 334.640µs | 340.734µs ± 2.869µs | 340.097µs ± 1.566µs | 342.068µs | 345.822µs | 349.078µs | 352.280µs | 3.58% | 1.018 | 1.567 | 0.84% | 0.203µs | 1 | 200 |
| trace_buffer/1_senders/no_delay | throughput | 2554786.710op/s | 2641540.774op/s ± 22060.197op/s | 2646307.041op/s ± 12242.303op/s | 2656196.210op/s | 2670472.932op/s | 2683647.112op/s | 2689454.030op/s | 1.63% | -0.955 | 1.394 | 0.83% | 1559.892op/s | 1 | 200 |
| trace_buffer/2_senders/10us_delay | execution_time | 56.582ms | 56.949ms ± 0.192ms | 56.935ms ± 0.126ms | 57.080ms | 57.309ms | 57.379ms | 57.490ms | 0.98% | 0.540 | -0.323 | 0.34% | 0.014ms | 1 | 200 |
| trace_buffer/2_senders/10us_delay | throughput | 31309.795op/s | 31607.785op/s ± 106.606op/s | 31615.189op/s ± 70.182op/s | 31683.756op/s | 31749.854op/s | 31790.370op/s | 31812.468op/s | 0.62% | -0.526 | -0.340 | 0.34% | 7.538op/s | 1 | 200 |
| trace_buffer/2_senders/1us_delay | execution_time | 50.069ms | 50.109ms ± 0.034ms | 50.103ms ± 0.018ms | 50.118ms | 50.179ms | 50.220ms | 50.286ms | 0.36% | 1.795 | 4.230 | 0.07% | 0.002ms | 1 | 200 |
| trace_buffer/2_senders/1us_delay | throughput | 35795.606op/s | 35921.798op/s ± 24.549op/s | 35925.686op/s ± 12.923op/s | 35939.943op/s | 35947.053op/s | 35949.849op/s | 35950.679op/s | 0.07% | -1.788 | 4.195 | 0.07% | 1.736op/s | 1 | 200 |
| trace_buffer/2_senders/no_delay | execution_time | 861.340µs | 909.718µs ± 16.764µs | 911.392µs ± 11.044µs | 921.227µs | 933.483µs | 944.402µs | 965.478µs | 5.93% | -0.282 | 0.340 | 1.84% | 1.185µs | 1 | 200 |
| trace_buffer/2_senders/no_delay | throughput | 1864361.273op/s | 1979307.899op/s ± 36706.369op/s | 1975000.541op/s ± 23754.178op/s | 2002272.755op/s | 2047476.006op/s | 2074029.021op/s | 2089767.511op/s | 5.81% | 0.405 | 0.370 | 1.85% | 2595.532op/s | 1 | 200 |
| trace_buffer/4_senders/10us_delay | execution_time | 57.033ms | 57.438ms ± 0.166ms | 57.420ms ± 0.112ms | 57.538ms | 57.753ms | 57.871ms | 57.919ms | 0.87% | 0.419 | 0.094 | 0.29% | 0.012ms | 1 | 200 |
| trace_buffer/4_senders/10us_delay | throughput | 62156.027op/s | 62676.428op/s ± 180.471op/s | 62695.547op/s ± 122.090op/s | 62798.668op/s | 62912.537op/s | 63042.635op/s | 63121.385op/s | 0.68% | -0.402 | 0.076 | 0.29% | 12.761op/s | 1 | 200 |
| trace_buffer/4_senders/1us_delay | execution_time | 50.112ms | 50.174ms ± 0.047ms | 50.161ms ± 0.021ms | 50.194ms | 50.268ms | 50.322ms | 50.394ms | 0.47% | 1.579 | 3.035 | 0.09% | 0.003ms | 1 | 200 |
| trace_buffer/4_senders/1us_delay | throughput | 71436.408op/s | 71750.797op/s ± 66.529op/s | 71769.246op/s ± 29.711op/s | 71793.977op/s | 71825.945op/s | 71836.315op/s | 71839.238op/s | 0.10% | -1.572 | 3.001 | 0.09% | 4.704op/s | 1 | 200 |
| trace_buffer/4_senders/no_delay | execution_time | 2.347ms | 2.493ms ± 0.049ms | 2.496ms ± 0.034ms | 2.528ms | 2.568ms | 2.597ms | 2.603ms | 4.29% | -0.235 | -0.248 | 1.98% | 0.003ms | 1 | 200 |
| trace_buffer/4_senders/no_delay | throughput | 1383034.242op/s | 1444511.614op/s ± 28791.406op/s | 1442421.179op/s ± 19346.613op/s | 1463488.575op/s | 1494523.656op/s | 1517736.134op/s | 1533770.896op/s | 6.33% | 0.337 | -0.142 | 1.99% | 2035.860op/s | 1 | 200 |
| trace_buffer/8_senders/10us_delay | execution_time | 57.548ms | 57.890ms ± 0.213ms | 57.815ms ± 0.124ms | 57.995ms | 58.347ms | 58.443ms | 58.554ms | 1.28% | 0.994 | 0.284 | 0.37% | 0.015ms | 1 | 200 |
| trace_buffer/8_senders/10us_delay | throughput | 122962.879op/s | 124375.977op/s ± 455.499op/s | 124535.423op/s ± 268.505op/s | 124716.313op/s | 124901.976op/s | 124985.632op/s | 125113.706op/s | 0.46% | -0.980 | 0.250 | 0.37% | 32.209op/s | 1 | 200 |
| trace_buffer/8_senders/1us_delay | execution_time | 50.195ms | 50.285ms ± 0.053ms | 50.270ms ± 0.029ms | 50.311ms | 50.393ms | 50.435ms | 50.467ms | 0.39% | 1.167 | 0.984 | 0.10% | 0.004ms | 1 | 200 |
| trace_buffer/8_senders/1us_delay | throughput | 142666.338op/s | 143182.703op/s ± 150.375op/s | 143227.545op/s ± 81.216op/s | 143287.680op/s | 143352.054op/s | 143381.368op/s | 143439.287op/s | 0.15% | -1.162 | 0.969 | 0.10% | 10.633op/s | 1 | 200 |
| trace_buffer/8_senders/no_delay | execution_time | 5.477ms | 6.367ms ± 0.235ms | 6.407ms ± 0.140ms | 6.543ms | 6.618ms | 6.660ms | 6.973ms | 8.83% | -1.388 | 2.810 | 3.68% | 0.017ms | 1 | 200 |
| trace_buffer/8_senders/no_delay | throughput | 1032574.425op/s | 1132481.763op/s ± 44264.706op/s | 1123719.342op/s ± 24016.235op/s | 1151111.580op/s | 1214837.659op/s | 1306108.234op/s | 1314543.636op/s | 16.98% | 1.728 | 4.125 | 3.90% | 3129.987op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| trace_buffer/1_senders/10us_delay | execution_time | [56.534ms; 56.571ms] or [-0.032%; +0.032%] | None | None | None |
| trace_buffer/1_senders/10us_delay | throughput | [15909.353op/s; 15919.672op/s] or [-0.032%; +0.032%] | None | None | None |
| trace_buffer/1_senders/1us_delay | execution_time | [50.073ms; 50.081ms] or [-0.008%; +0.008%] | None | None | None |
| trace_buffer/1_senders/1us_delay | throughput | [17970.879op/s; 17973.589op/s] or [-0.008%; +0.008%] | None | None | None |
| trace_buffer/1_senders/no_delay | execution_time | [340.336µs; 341.132µs] or [-0.117%; +0.117%] | None | None | None |
| trace_buffer/1_senders/no_delay | throughput | [2638483.442op/s; 2644598.105op/s] or [-0.116%; +0.116%] | None | None | None |
| trace_buffer/2_senders/10us_delay | execution_time | [56.922ms; 56.975ms] or [-0.047%; +0.047%] | None | None | None |
| trace_buffer/2_senders/10us_delay | throughput | [31593.011op/s; 31622.560op/s] or [-0.047%; +0.047%] | None | None | None |
| trace_buffer/2_senders/1us_delay | execution_time | [50.104ms; 50.114ms] or [-0.009%; +0.009%] | None | None | None |
| trace_buffer/2_senders/1us_delay | throughput | [35918.396op/s; 35925.200op/s] or [-0.009%; +0.009%] | None | None | None |
| trace_buffer/2_senders/no_delay | execution_time | [907.395µs; 912.041µs] or [-0.255%; +0.255%] | None | None | None |
| trace_buffer/2_senders/no_delay | throughput | [1974220.750op/s; 1984395.049op/s] or [-0.257%; +0.257%] | None | None | None |
| trace_buffer/4_senders/10us_delay | execution_time | [57.415ms; 57.461ms] or [-0.040%; +0.040%] | None | None | None |
| trace_buffer/4_senders/10us_delay | throughput | [62651.416op/s; 62701.439op/s] or [-0.040%; +0.040%] | None | None | None |
| trace_buffer/4_senders/1us_delay | execution_time | [50.167ms; 50.180ms] or [-0.013%; +0.013%] | None | None | None |
| trace_buffer/4_senders/1us_delay | throughput | [71741.577op/s; 71760.017op/s] or [-0.013%; +0.013%] | None | None | None |
| trace_buffer/4_senders/no_delay | execution_time | [2.486ms; 2.500ms] or [-0.275%; +0.275%] | None | None | None |
| trace_buffer/4_senders/no_delay | throughput | [1440521.402op/s; 1448501.826op/s] or [-0.276%; +0.276%] | None | None | None |
| trace_buffer/8_senders/10us_delay | execution_time | [57.860ms; 57.919ms] or [-0.051%; +0.051%] | None | None | None |
| trace_buffer/8_senders/10us_delay | throughput | [124312.849op/s; 124439.105op/s] or [-0.051%; +0.051%] | None | None | None |
| trace_buffer/8_senders/1us_delay | execution_time | [50.278ms; 50.293ms] or [-0.015%; +0.015%] | None | None | None |
| trace_buffer/8_senders/1us_delay | throughput | [143161.863op/s; 143203.544op/s] or [-0.015%; +0.015%] | None | None | None |
| trace_buffer/8_senders/no_delay | execution_time | [6.334ms; 6.399ms] or [-0.511%; +0.511%] | None | None | None |
| trace_buffer/8_senders/no_delay | throughput | [1126347.100op/s; 1138616.425op/s] or [-0.542%; +0.542%] | None | None | None |
Baseline
Baseline benchmark details
Group 1
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 407f8f2 | 1784125681 | main |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| trace_buffer/1_senders/10us_delay | execution_time | 56.351ms | 56.672ms ± 0.253ms | 56.600ms ± 0.132ms | 56.803ms | 57.130ms | 57.595ms | 57.611ms | 1.79% | 1.520 | 2.673 | 0.44% | 0.018ms | 1 | 200 |
| trace_buffer/1_senders/10us_delay | throughput | 15621.929op/s | 15881.127op/s ± 70.311op/s | 15901.057op/s ± 37.100op/s | 15933.859op/s | 15959.013op/s | 15970.575op/s | 15971.274op/s | 0.44% | -1.489 | 2.544 | 0.44% | 4.972op/s | 1 | 200 |
| trace_buffer/1_senders/1us_delay | execution_time | 49.978ms | 50.073ms ± 0.031ms | 50.076ms ± 0.016ms | 50.088ms | 50.135ms | 50.153ms | 50.203ms | 0.25% | 0.678 | 1.861 | 0.06% | 0.002ms | 1 | 200 |
| trace_buffer/1_senders/1us_delay | throughput | 17927.383op/s | 17973.753op/s ± 11.251op/s | 17972.608op/s ± 5.823op/s | 17983.874op/s | 17988.824op/s | 17990.365op/s | 18008.037op/s | 0.20% | -0.672 | 1.844 | 0.06% | 0.796op/s | 1 | 200 |
| trace_buffer/1_senders/no_delay | execution_time | 331.685µs | 339.895µs ± 3.456µs | 339.286µs ± 1.804µs | 341.540µs | 345.524µs | 351.366µs | 353.697µs | 4.25% | 1.176 | 2.380 | 1.01% | 0.244µs | 1 | 200 |
| trace_buffer/1_senders/no_delay | throughput | 2544553.980op/s | 2648148.544op/s ± 26619.843op/s | 2652625.440op/s ± 14176.693op/s | 2665092.719op/s | 2680926.150op/s | 2693423.401op/s | 2713414.285op/s | 2.29% | -1.087 | 2.096 | 1.00% | 1882.307op/s | 1 | 200 |
| trace_buffer/2_senders/10us_delay | execution_time | 56.765ms | 57.193ms ± 0.219ms | 57.174ms ± 0.158ms | 57.357ms | 57.555ms | 57.639ms | 57.879ms | 1.23% | 0.290 | -0.387 | 0.38% | 0.015ms | 1 | 200 |
| trace_buffer/2_senders/10us_delay | throughput | 31099.628op/s | 31473.007op/s ± 120.458op/s | 31482.800op/s ± 86.977op/s | 31563.492op/s | 31668.742op/s | 31698.906op/s | 31709.904op/s | 0.72% | -0.272 | -0.408 | 0.38% | 8.518op/s | 1 | 200 |
| trace_buffer/2_senders/1us_delay | execution_time | 50.056ms | 50.115ms ± 0.038ms | 50.108ms ± 0.021ms | 50.130ms | 50.193ms | 50.234ms | 50.278ms | 0.34% | 1.444 | 2.437 | 0.08% | 0.003ms | 1 | 200 |
| trace_buffer/2_senders/1us_delay | throughput | 35800.962op/s | 35917.213op/s ± 27.406op/s | 35922.761op/s ± 15.200op/s | 35937.609op/s | 35949.581op/s | 35953.016op/s | 35960.066op/s | 0.10% | -1.439 | 2.416 | 0.08% | 1.938op/s | 1 | 200 |
| trace_buffer/2_senders/no_delay | execution_time | 860.084µs | 912.773µs ± 18.276µs | 914.017µs ± 12.032µs | 925.678µs | 940.489µs | 957.054µs | 965.021µs | 5.58% | -0.206 | 0.120 | 2.00% | 1.292µs | 1 | 200 |
| trace_buffer/2_senders/no_delay | throughput | 1865244.276op/s | 1972803.102op/s ± 39712.430op/s | 1969328.514op/s ± 25983.623op/s | 1996327.150op/s | 2047638.740op/s | 2065276.629op/s | 2092819.922op/s | 6.27% | 0.330 | 0.166 | 2.01% | 2808.093op/s | 1 | 200 |
| trace_buffer/4_senders/10us_delay | execution_time | 57.115ms | 57.407ms ± 0.143ms | 57.381ms ± 0.085ms | 57.495ms | 57.651ms | 57.814ms | 57.861ms | 0.84% | 0.552 | 0.282 | 0.25% | 0.010ms | 1 | 200 |
| trace_buffer/4_senders/10us_delay | throughput | 62218.413op/s | 62710.816op/s ± 156.159op/s | 62738.326op/s ± 93.264op/s | 62819.655op/s | 62934.767op/s | 63005.103op/s | 63030.215op/s | 0.47% | -0.537 | 0.257 | 0.25% | 11.042op/s | 1 | 200 |
| trace_buffer/4_senders/1us_delay | execution_time | 50.112ms | 50.164ms ± 0.034ms | 50.155ms ± 0.015ms | 50.174ms | 50.242ms | 50.278ms | 50.283ms | 0.26% | 1.540 | 2.197 | 0.07% | 0.002ms | 1 | 200 |
| trace_buffer/4_senders/1us_delay | throughput | 71595.184op/s | 71765.005op/s ± 49.009op/s | 71778.050op/s ± 22.113op/s | 71795.890op/s | 71819.402op/s | 71824.995op/s | 71839.369op/s | 0.09% | -1.537 | 2.184 | 0.07% | 3.465op/s | 1 | 200 |
| trace_buffer/4_senders/no_delay | execution_time | 2.090ms | 2.353ms ± 0.081ms | 2.359ms ± 0.055ms | 2.414ms | 2.471ms | 2.510ms | 2.521ms | 6.87% | -0.470 | 0.001 | 3.44% | 0.006ms | 1 | 200 |
| trace_buffer/4_senders/no_delay | throughput | 1428198.510op/s | 1531972.597op/s ± 53920.475op/s | 1526263.405op/s ± 35438.253op/s | 1561780.391op/s | 1624959.869op/s | 1683831.982op/s | 1722085.017op/s | 12.83% | 0.662 | 0.359 | 3.51% | 3812.753op/s | 1 | 200 |
| trace_buffer/8_senders/10us_delay | execution_time | 57.615ms | 57.907ms ± 0.241ms | 57.817ms ± 0.095ms | 57.981ms | 58.439ms | 58.528ms | 58.705ms | 1.54% | 1.348 | 0.901 | 0.42% | 0.017ms | 1 | 200 |
| trace_buffer/8_senders/10us_delay | throughput | 122646.896op/s | 124339.057op/s ± 514.598op/s | 124530.255op/s ± 204.987op/s | 124688.809op/s | 124852.397op/s | 124900.027op/s | 124967.771op/s | 0.35% | -1.335 | 0.860 | 0.41% | 36.388op/s | 1 | 200 |
| trace_buffer/8_senders/1us_delay | execution_time | 50.199ms | 50.279ms ± 0.048ms | 50.267ms ± 0.026ms | 50.301ms | 50.364ms | 50.459ms | 50.494ms | 0.45% | 1.601 | 3.497 | 0.10% | 0.003ms | 1 | 200 |
| trace_buffer/8_senders/1us_delay | throughput | 142592.165op/s | 143201.164op/s ± 137.850op/s | 143234.767op/s ± 73.488op/s | 143291.281op/s | 143364.425op/s | 143383.849op/s | 143429.612op/s | 0.14% | -1.592 | 3.460 | 0.10% | 9.747op/s | 1 | 200 |
| trace_buffer/8_senders/no_delay | execution_time | 5.651ms | 6.506ms ± 0.176ms | 6.552ms ± 0.104ms | 6.621ms | 6.711ms | 6.764ms | 6.795ms | 3.71% | -1.501 | 3.730 | 2.70% | 0.012ms | 1 | 200 |
| trace_buffer/8_senders/no_delay | throughput | 1059585.444op/s | 1107472.427op/s ± 31414.332op/s | 1098854.504op/s ± 17405.882op/s | 1122030.027op/s | 1160597.390op/s | 1222222.189op/s | 1274134.226op/s | 15.95% | 1.816 | 5.445 | 2.83% | 2221.329op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| trace_buffer/1_senders/10us_delay | execution_time | [56.637ms; 56.707ms] or [-0.062%; +0.062%] | None | None | None |
| trace_buffer/1_senders/10us_delay | throughput | [15871.383op/s; 15890.872op/s] or [-0.061%; +0.061%] | None | None | None |
| trace_buffer/1_senders/1us_delay | execution_time | [50.069ms; 50.077ms] or [-0.009%; +0.009%] | None | None | None |
| trace_buffer/1_senders/1us_delay | throughput | [17972.194op/s; 17975.313op/s] or [-0.009%; +0.009%] | None | None | None |
| trace_buffer/1_senders/no_delay | execution_time | [339.416µs; 340.374µs] or [-0.141%; +0.141%] | None | None | None |
| trace_buffer/1_senders/no_delay | throughput | [2644459.290op/s; 2651837.798op/s] or [-0.139%; +0.139%] | None | None | None |
| trace_buffer/2_senders/10us_delay | execution_time | [57.162ms; 57.223ms] or [-0.053%; +0.053%] | None | None | None |
| trace_buffer/2_senders/10us_delay | throughput | [31456.313op/s; 31489.702op/s] or [-0.053%; +0.053%] | None | None | None |
| trace_buffer/2_senders/1us_delay | execution_time | [50.110ms; 50.121ms] or [-0.011%; +0.011%] | None | None | None |
| trace_buffer/2_senders/1us_delay | throughput | [35913.415op/s; 35921.011op/s] or [-0.011%; +0.011%] | None | None | None |
| trace_buffer/2_senders/no_delay | execution_time | [910.240µs; 915.306µs] or [-0.277%; +0.277%] | None | None | None |
| trace_buffer/2_senders/no_delay | throughput | [1967299.341op/s; 1978306.863op/s] or [-0.279%; +0.279%] | None | None | None |
| trace_buffer/4_senders/10us_delay | execution_time | [57.387ms; 57.427ms] or [-0.035%; +0.035%] | None | None | None |
| trace_buffer/4_senders/10us_delay | throughput | [62689.174op/s; 62732.458op/s] or [-0.035%; +0.035%] | None | None | None |
| trace_buffer/4_senders/1us_delay | execution_time | [50.159ms; 50.169ms] or [-0.009%; +0.009%] | None | None | None |
| trace_buffer/4_senders/1us_delay | throughput | [71758.213op/s; 71771.797op/s] or [-0.009%; +0.009%] | None | None | None |
| trace_buffer/4_senders/no_delay | execution_time | [2.341ms; 2.364ms] or [-0.478%; +0.478%] | None | None | None |
| trace_buffer/4_senders/no_delay | throughput | [1524499.738op/s; 1539445.457op/s] or [-0.488%; +0.488%] | None | None | None |
| trace_buffer/8_senders/10us_delay | execution_time | [57.874ms; 57.941ms] or [-0.058%; +0.058%] | None | None | None |
| trace_buffer/8_senders/10us_delay | throughput | [124267.739op/s; 124410.375op/s] or [-0.057%; +0.057%] | None | None | None |
| trace_buffer/8_senders/1us_delay | execution_time | [50.272ms; 50.286ms] or [-0.013%; +0.013%] | None | None | None |
| trace_buffer/8_senders/1us_delay | throughput | [143182.059op/s; 143220.268op/s] or [-0.013%; +0.013%] | None | None | None |
| trace_buffer/8_senders/no_delay | execution_time | [6.482ms; 6.531ms] or [-0.376%; +0.376%] | None | None | None |
| trace_buffer/8_senders/no_delay | throughput | [1103118.703op/s; 1111826.151op/s] or [-0.393%; +0.393%] | None | None | None |
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 4880dd4 | Docs | Datadog PR Page | Give us feedback! |
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
What does this PR do?
Adds a dedicated
set_otlp_timeoutsetter toTraceExporterBuilder(and the FFI equivalentddog_trace_exporter_config_set_otlp_timeout) that sets the OTLP trace-export request timeout independently of the agent connection timeout.Before this change,
set_connection_timeoutwas the only timeout knob, and its value was wired into three places at build time:Endpoint.timeout_ms(agent info, stats, Remote Config, telemetry, agent-path trace sends),After this change:
otlp_timeout→connection_timeout→ default.Endpoint.timeout_msis driven solely byconnection_timeout.connection_timeout(unchanged), so a traces-scopedotlp_timeoutnever shortens OTLP metrics/stats requests.Timeout resolution is centralized in a small
resolve_otlp_timeoutshelper.Motivation
A host tracer that maps
OTEL_EXPORTER_OTLP_TRACES_TIMEOUTontoset_connection_timeout(the only available knob) silently reshapes agent-side request timeouts. A short OTLP timeout tuned for a fast collector shortens/info, Remote Config, and telemetry requests — disabling client-side stats, blocking RC, and dropping telemetry — while the OTLP default (10s) conversely loosens the agent's 3s default. The two concerns need separate knobs;OtlpTraceConfigalready carries its owntimeoutfield, so only the builder's input plumbing was conflated.This was surfaced while wiring OTLP trace export in dd-trace-rs (DataDog/dd-trace-rs#251), which will call
set_otlp_timeoutonce a release containing this change ships.Additional Notes
connection_timeoutare unaffected — the OTLP trace timeout still falls back to it, and the agent endpoint timeout is unchanged.set_connection_timeoutconventions; the C header is cbindgen-generated at build time (no checked-in header to edit).connection_timeoutto avoid a regression where a traces-scoped timeout would govern OTLP metrics/stats export.How to test the change?
New unit tests (all passing):
resolve_otlp_timeouts_scopes_trace_and_metrics_independently(builder) — asserts the(trace, metrics)timeout resolution across allotlp_timeout/connection_timeoutcombinations, proving metrics never picks up the traces-scoped value.test_otlp_timeout_decoupled_from_agent_connection_timeout(trace_exporter) — builds exporters and asserts the agentEndpoint.timeout_msand OTLP config timeout independently across the four combinations.config_otlp_timeout_test(FFI) — asserts the setter stores the OTLP timeout and leavesconnection_timeoutuntouched.The existing
test_connection_timeoutand OTLP export integration tests continue to pass;cargo fmtandcargo clippyare clean.🤖 Generated with Claude Code