perf(profiling): reduce ProfilesDictionary arena footprint by taegyunkim · Pull Request #2246 · DataDog/libdatadog

taegyunkim · 2026-07-16T19:11:25Z

What does this PR do?

Reduces the memory floor of the profiling ProfilesDictionary arena-backed storage by changing existing shard count and arena chunk size constants.

Specifically:

ParallelSliceSet / ParallelStringSet shard count changes from 16 to 4.
SliceSet arena size hint changes from 1 MiB to 256 KiB.
Set<T> arena size hint changes from 1 MiB to 256 KiB, reducing the per-shard floor for function and mapping sets.
string shard selection now derives from N_SHARDS instead of hardcoding a 4-bit mask.

This changes the approximate ProfilesDictionary arena floor from:

functions: 4 * 1 MiB   = 4 MiB
mappings:  2 * 1 MiB   = 2 MiB
strings:   16 * 1 MiB  = 16 MiB
--------------------------------
total:                  22 MiB

to:

functions: 4 * 256 KiB = 1 MiB
mappings:  2 * 256 KiB = 512 KiB
strings:   4 * 256 KiB = 1 MiB
--------------------------------
total:                  2.5 MiB

No allocator APIs are added or changed in this PR.

Motivation

In Python profiling memory experiments, the profiles dictionary storage showed up as a meaningful virtual memory floor. The goal is to make the dictionary storage floor smaller by reducing the string shard multiplicity and reducing arena chunk sizes for strings, functions, and mappings.

This PR intentionally leaves the profile-local StringTable chunk size unchanged. That can be evaluated separately if needed.

Additional Notes

The main trade-off is potentially higher string insertion contention because the string set now has 4 shards instead of 16. For Python profiling, this should be acceptable, but we should validate with workload benchmarks and profile counts/sizes.

Function shards remain at 4. Mapping shards remain at 2. This PR only reduces their per-shard arena size.

How to test the change?

Ran:

cargo fmt
cargo test -p libdd-profiling --lib

Final result:

141 passed, 0 failed

github-actions · 2026-07-16T19:13:52Z

📚 Documentation Check Results

⚠️ 650 documentation warning(s) found

📦 `libdd-profiling` - 650 warning(s)

Updated: 2026-07-16 21:14:54 UTC | Commit: 44b0190 | missing-docs job results

github-actions · 2026-07-16T19:15:34Z

🔒 Cargo Deny Results

⚠️ 4 issue(s) found, showing only errors (advisories, bans, sources)

📦 `libdd-profiling` - 4 error(s)

Show output

error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:48:1
   │
48 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     ├── crossbeam-deque v0.8.5
     │   └── rayon-core v1.12.1
     │       └── rayon v1.10.0
     │           └── criterion v0.5.1
     │               └── (dev) libdd-profiling v1.0.0
     │                   └── (dev) libdd-profiling v1.0.0 (*)
     └── moka v0.12.13
         └── hickory-resolver v0.25.2
             └── reqwest v0.13.2
                 ├── libdd-common v5.1.0
                 │   └── libdd-profiling v1.0.0 (*)
                 └── libdd-profiling v1.0.0 (*)

error[vulnerability]: NSEC3 closest-encloser proof validation enters unbounded loop on cross-zone responses
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:79:1
   │
79 │ hickory-proto 0.25.2 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0118
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0118
   ├ The NSEC3 closest-encloser proof validation in `hickory-proto`'s
     `DnssecDnsHandle` walks from the QNAME up to the SOA owner name, building a
     list of candidate encloser names. The iterator used assumes the
     QNAME is a descendant of the SOA owner, terminating only when the current
     candidate equals the SOA name. When the SOA in a response's authority section
     is not an ancestor of the QNAME, the loop stalls at the DNS root and never
     terminates, repeatedly calling `Name::base_name()` and pushing newly allocated
     `Name` and hashed-name entries into the candidate `Vec`.
     
     The bug is reachable by any caller of `DnssecDnsHandle` — including the
     resolver, recursor, and client — when built with the `dnssec-ring` or
     `dnssec-aws-lc-rs` feature and configured to perform DNSSEC validation. It is
     triggered while validating a NoData or NXDomain response whose authority
     section contains an SOA record from a zone other than an ancestor of the
     QNAME, on a code path that requires NSEC3 closest-encloser proof. In practice
     this can be reached through an insecure CNAME chain that crosses zone
     boundaries into a DNSSEC-signed zone returning NoData, but the minimum
     condition is just a mismatched SOA owner on a response requiring NSEC3
     validation.
     
     A `debug_assert_ne!(name, Name::root())` guards the loop body, so debug builds
     abort with a panic on the first iteration past the root. Release builds
     compile the assertion out and run the loop unbounded, allocating until the
     process exhausts available memory (OOM). A reachable upstream attacker who
     can return such a response can therefore crash a debug-built validator or
     exhaust memory on a release-built one.
     
     The affected code was migrated from `hickory-proto` to `hickory-net` as part of
     the 0.26.0 release. The `hickory-proto` 0.26.x release no longer offers
     `DnssecDnsHandle` and so we recommend all affected users update to `hickory-net`
     0.26.1 when the implementation of that type is required.
   ├ Announcement: https://github.com/hickory-dns/hickory-dns/security/advisories/GHSA-3v94-mw7p-v465
   ├ Solution: No safe upgrade is available!
   ├ hickory-proto v0.25.2
     └── hickory-resolver v0.25.2
         └── reqwest v0.13.2
             ├── libdd-common v5.1.0
             │   └── libdd-profiling v1.0.0
             │       └── (dev) libdd-profiling v1.0.0 (*)
             └── libdd-profiling v1.0.0 (*)

error[vulnerability]: CPU exhaustion during message encoding due to O(n²) name compression
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:79:1
   │
79 │ hickory-proto 0.25.2 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0119
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0119
   ├ During message encoding, `hickory-proto`'s `BinEncoder` stores pointers to
     labels that are candidates for name compression in a `Vec<(usize, Vec<u8>)>`.
     The name compression logic then searches for matches with a linear scan.
     
     A malicious message with many records can both introduce many candidate labels,
     and invoke this linear scan many times. This can amplify CPU exhaustion in DoS
     attacks.
     
     This is similar to
     [CVE-2024-8508](https://www.nlnetlabs.nl/downloads/unbound/CVE-2024-8508.txt).
     
     We recommend all affected users update to `hickory-proto` 0.26.1 for the fix.
   ├ Announcement: https://github.com/hickory-dns/hickory-dns/security/advisories/GHSA-q2qq-hmj6-3wpp
   ├ Solution: Upgrade to >=0.26.1 (try `cargo update -p hickory-proto`)
   ├ hickory-proto v0.25.2
     └── hickory-resolver v0.25.2
         └── reqwest v0.13.2
             ├── libdd-common v5.1.0
             │   └── libdd-profiling v1.0.0
             │       └── (dev) libdd-profiling v1.0.0 (*)
             └── libdd-profiling v1.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:156:1
    │
156 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by @lopopolo) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   └── libdd-profiling v1.0.0
      │       └── (dev) libdd-profiling v1.0.0 (*)
      ├── libdd-profiling v1.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-profiling v1.0.0 (*)

advisories FAILED, bans ok, sources ok

Updated: 2026-07-16 21:16:35 UTC | Commit: 44b0190 | dependency-check job results

pr-commenter · 2026-07-16T19:22:33Z

Benchmarks

Comparison

Benchmark execution time: 2026-07-16 21:22:14

Comparing candidate commit 6b4ba8c in PR branch taegyun/reduce-profile-string-shards with baseline commit 0c6e2a5 in branch main.

Found 4 performance improvements and 4 performance regressions! Performance is the same for 5 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:profiles_dictionary/profile_string_inserts/threads/1

🟩 execution_time [-131.201µs; -128.754µs] or [-40.941%; -40.177%]
🟩 throughput [+2165306.069op/s; +2195122.922op/s] or [+67.763%; +68.696%]

scenario:profiles_dictionary/profile_string_inserts/threads/16

🟥 execution_time [+2.865ms; +2.897ms] or [+146.443%; +148.080%]
🟥 throughput [-5013185.443op/s; -4962478.318op/s] or [-59.860%; -59.254%]

scenario:profiles_dictionary/profile_string_inserts/threads/2

🟩 execution_time [-42.616µs; -40.584µs] or [-7.517%; -7.159%]
🟩 throughput [+279285.342op/s; +292887.286op/s] or [+7.731%; +8.107%]

scenario:profiles_dictionary/profile_string_inserts/threads/4

🟥 execution_time [+208.224µs; +213.066µs] or [+27.678%; +28.322%]
🟥 throughput [-1206953.586op/s; -1175227.438op/s] or [-22.167%; -21.585%]

Candidate

Candidate benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`6b4ba8c`	1784236353	taegyun/reduce-profile-string-shards

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
profile_serialize_compressed_pprof_timestamped_x1000	execution_time	977.869µs	981.691µs ± 2.655µs	980.904µs ± 0.843µs	982.320µs	985.186µs	990.494µs	1004.529µs	2.41%	4.154	28.467	0.27%	0.188µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
profile_serialize_compressed_pprof_timestamped_x1000	execution_time	[981.323µs; 982.059µs] or [-0.037%; +0.037%]	None	None	None

Group 2

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`6b4ba8c`	1784236353	taegyun/reduce-profile-string-shards

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching string interning on wordpress profile	execution_time	113.156µs	114.024µs ± 0.413µs	113.986µs ± 0.193µs	114.197µs	114.414µs	115.173µs	118.010µs	3.53%	4.774	43.149	0.36%	0.029µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching string interning on wordpress profile	execution_time	[113.967µs; 114.081µs] or [-0.050%; +0.050%]	None	None	None

Group 3

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`6b4ba8c`	1784236353	taegyun/reduce-profile-string-shards

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
profile_add_sample2_frames_x1000	execution_time	721.862µs	723.252µs ± 0.487µs	723.252µs ± 0.280µs	723.509µs	724.139µs	724.511µs	725.309µs	0.28%	0.540	1.600	0.07%	0.034µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
profile_add_sample2_frames_x1000	execution_time	[723.185µs; 723.320µs] or [-0.009%; +0.009%]	None	None	None

Group 4

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`6b4ba8c`	1784236353	taegyun/reduce-profile-string-shards

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
profile_add_sample_frames_x1000	execution_time	4.138ms	4.142ms ± 0.003ms	4.142ms ± 0.001ms	4.144ms	4.146ms	4.148ms	4.169ms	0.66%	4.476	39.639	0.07%	0.000ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
profile_add_sample_frames_x1000	execution_time	[4.142ms; 4.143ms] or [-0.009%; +0.009%]	None	None	None

Group 5

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`6b4ba8c`	1784236353	taegyun/reduce-profile-string-shards

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
profile_add_sample_timestamped_x1000	execution_time	4.111ms	4.118ms ± 0.008ms	4.117ms ± 0.002ms	4.118ms	4.121ms	4.124ms	4.227ms	2.69%	11.942	155.972	0.20%	0.001ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
profile_add_sample_timestamped_x1000	execution_time	[4.117ms; 4.119ms] or [-0.028%; +0.028%]	None	None	None

Group 6

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`6b4ba8c`	1784236353	taegyun/reduce-profile-string-shards

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
profiles_dictionary/profile_string_inserts/threads/1	execution_time	189.453µs	190.490µs ± 0.520µs	190.549µs ± 0.301µs	190.887µs	191.047µs	191.088µs	191.098µs	0.29%	-0.795	-0.324	0.26%	0.164µs	1	10
profiles_dictionary/profile_string_inserts/threads/1	throughput	5358504.654op/s	5375646.012op/s ± 14705.737op/s	5373957.989op/s ± 8480.436op/s	5379438.951op/s	5400197.271op/s	5404070.225op/s	5405038.464op/s	0.58%	0.804	-0.312	0.26%	4650.362op/s	1	10
profiles_dictionary/profile_string_inserts/threads/16	execution_time	4.782ms	4.837ms ± 0.024ms	4.836ms ± 0.008ms	4.847ms	4.866ms	4.877ms	4.880ms	0.90%	-0.763	1.523	0.48%	0.008ms	1	10
profiles_dictionary/profile_string_inserts/threads/16	throughput	3357631.877op/s	3387086.951op/s ± 17122.665op/s	3387949.540op/s ± 5687.335op/s	3390953.378op/s	3410885.009op/s	3423368.884op/s	3426489.852op/s	1.14%	0.805	1.560	0.48%	5414.662op/s	1	10
profiles_dictionary/profile_string_inserts/threads/2	execution_time	523.707µs	525.298µs ± 0.874µs	525.478µs ± 0.431µs	525.655µs	526.428µs	526.799µs	526.891µs	0.27%	-0.120	-0.063	0.16%	0.276µs	1	10
profiles_dictionary/profile_string_inserts/threads/2	throughput	3886949.995op/s	3898746.325op/s ± 6489.527op/s	3897406.069op/s ± 3200.403op/s	3901316.471op/s	3908495.470op/s	3910164.953op/s	3910582.324op/s	0.34%	0.129	-0.065	0.16%	2052.169op/s	1	10
profiles_dictionary/profile_string_inserts/threads/4	execution_time	960.622µs	962.947µs ± 2.080µs	962.445µs ± 0.637µs	962.865µs	966.633µs	967.588µs	967.827µs	0.56%	1.423	1.137	0.20%	0.658µs	1	10
profiles_dictionary/profile_string_inserts/threads/4	throughput	4232162.152op/s	4253627.465op/s ± 9161.366op/s	4255828.831op/s ± 2817.970op/s	4258853.132op/s	4261834.992op/s	4263488.977op/s	4263902.473op/s	0.19%	-1.416	1.121	0.20%	2897.078op/s	1	10

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
profiles_dictionary/profile_string_inserts/threads/1	execution_time	[190.168µs; 190.812µs] or [-0.169%; +0.169%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/1	throughput	[5366531.470op/s; 5384760.555op/s] or [-0.170%; +0.170%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/16	execution_time	[4.822ms; 4.852ms] or [-0.312%; +0.312%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/16	throughput	[3376474.409op/s; 3397699.494op/s] or [-0.313%; +0.313%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/2	execution_time	[524.757µs; 525.840µs] or [-0.103%; +0.103%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/2	throughput	[3894724.148op/s; 3902768.502op/s] or [-0.103%; +0.103%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/4	execution_time	[961.658µs; 964.236µs] or [-0.134%; +0.134%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/4	throughput	[4247949.295op/s; 4259305.634op/s] or [-0.133%; +0.133%]	None	None	None

Baseline

Baseline benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`0c6e2a5`	1784223782	main

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
profile_serialize_compressed_pprof_timestamped_x1000	execution_time	939.829µs	944.623µs ± 2.504µs	943.878µs ± 0.845µs	945.314µs	949.130µs	951.744µs	961.520µs	1.87%	2.288	9.944	0.26%	0.177µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
profile_serialize_compressed_pprof_timestamped_x1000	execution_time	[944.276µs; 944.970µs] or [-0.037%; +0.037%]	None	None	None

Group 2

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`0c6e2a5`	1784223782	main

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching string interning on wordpress profile	execution_time	113.218µs	114.272µs ± 0.384µs	114.252µs ± 0.125µs	114.392µs	114.714µs	115.485µs	117.054µs	2.45%	2.597	18.907	0.34%	0.027µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching string interning on wordpress profile	execution_time	[114.219µs; 114.325µs] or [-0.047%; +0.047%]	None	None	None

Group 3

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`0c6e2a5`	1784223782	main

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
profile_add_sample2_frames_x1000	execution_time	694.570µs	695.661µs ± 0.402µs	695.664µs ± 0.224µs	695.884µs	696.284µs	696.854µs	697.221µs	0.22%	0.134	1.512	0.06%	0.028µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
profile_add_sample2_frames_x1000	execution_time	[695.605µs; 695.717µs] or [-0.008%; +0.008%]	None	None	None

Group 4

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`0c6e2a5`	1784223782	main

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
profile_add_sample_frames_x1000	execution_time	4.058ms	4.063ms ± 0.002ms	4.062ms ± 0.001ms	4.063ms	4.066ms	4.069ms	4.071ms	0.21%	1.141	1.971	0.05%	0.000ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
profile_add_sample_frames_x1000	execution_time	[4.062ms; 4.063ms] or [-0.007%; +0.007%]	None	None	None

Group 5

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`0c6e2a5`	1784223782	main

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
profile_add_sample_timestamped_x1000	execution_time	4.063ms	4.070ms ± 0.006ms	4.069ms ± 0.002ms	4.071ms	4.074ms	4.081ms	4.150ms	1.99%	10.592	130.777	0.15%	0.000ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
profile_add_sample_timestamped_x1000	execution_time	[4.069ms; 4.071ms] or [-0.021%; +0.021%]	None	None	None

Group 6

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`0c6e2a5`	1784223782	main

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
profiles_dictionary/profile_string_inserts/threads/1	execution_time	317.357µs	320.468µs ± 1.904µs	320.981µs ± 1.105µs	321.868µs	322.601µs	322.873µs	322.941µs	0.61%	-0.435	-1.168	0.56%	0.602µs	1	10
profiles_dictionary/profile_string_inserts/threads/1	throughput	3170852.937op/s	3195431.516op/s ± 19034.880op/s	3190227.972op/s ± 10946.882op/s	3210266.130op/s	3223625.531op/s	3226044.741op/s	3226649.544op/s	1.14%	0.446	-1.162	0.57%	6019.358op/s	1	10
profiles_dictionary/profile_string_inserts/threads/16	execution_time	1.937ms	1.956ms ± 0.009ms	1.960ms ± 0.003ms	1.962ms	1.963ms	1.964ms	1.964ms	0.21%	-1.334	0.581	0.42%	0.003ms	1	10
profiles_dictionary/profile_string_inserts/threads/16	throughput	8342968.933op/s	8374918.832op/s ± 37150.304op/s	8360087.426op/s ± 14031.559op/s	8381235.793op/s	8440914.968op/s	8455207.536op/s	8458780.678op/s	1.18%	1.344	0.608	0.42%	11747.958op/s	1	10
profiles_dictionary/profile_string_inserts/threads/2	execution_time	564.312µs	566.898µs ± 1.387µs	566.917µs ± 0.858µs	567.914µs	568.615µs	568.909µs	568.983µs	0.36%	-0.382	-0.500	0.23%	0.439µs	1	10
profiles_dictionary/profile_string_inserts/threads/2	throughput	3599406.076op/s	3612660.012op/s ± 8848.242op/s	3612522.130op/s ± 5460.676op/s	3615695.329op/s	3626359.540op/s	3628629.218op/s	3629196.638op/s	0.46%	0.391	-0.493	0.23%	2798.060op/s	1	10
profiles_dictionary/profile_string_inserts/threads/4	execution_time	747.680µs	752.302µs ± 3.305µs	751.748µs ± 2.266µs	755.059µs	756.948µs	757.197µs	757.259µs	0.73%	0.254	-1.211	0.42%	1.045µs	1	10
profiles_dictionary/profile_string_inserts/threads/4	throughput	5408979.404op/s	5444717.977op/s ± 23898.233op/s	5448634.982op/s ± 16478.447op/s	5459513.178op/s	5474835.713op/s	5477592.232op/s	5478281.362op/s	0.54%	-0.245	-1.211	0.42%	7557.285op/s	1	10

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
profiles_dictionary/profile_string_inserts/threads/1	execution_time	[319.287µs; 321.648µs] or [-0.368%; +0.368%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/1	throughput	[3183633.792op/s; 3207229.240op/s] or [-0.369%; +0.369%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/16	execution_time	[1.951ms; 1.962ms] or [-0.273%; +0.273%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/16	throughput	[8351893.258op/s; 8397944.406op/s] or [-0.275%; +0.275%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/2	execution_time	[566.039µs; 567.758µs] or [-0.152%; +0.152%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/2	throughput	[3607175.915op/s; 3618144.108op/s] or [-0.152%; +0.152%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/4	execution_time	[750.253µs; 754.351µs] or [-0.272%; +0.272%]	None	None	None
profiles_dictionary/profile_string_inserts/threads/4	throughput	[5429905.971op/s; 5459529.983op/s] or [-0.272%; +0.272%]	None	None	None

datadog-datadog-prod-us1-2 · 2026-07-16T19:30:31Z

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 74.70% (-0.02%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 6b4ba8c | Docs | Datadog PR Page | Give us feedback!}

dd-octo-sts · 2026-07-16T19:56:56Z

Artifact Size Benchmark Report

aarch64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so	7.88 MB	7.88 MB	0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a	86.33 MB	86.33 MB	-0% (-120 B) 👌

aarch64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.62 MB	10.62 MB	0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a	97.54 MB	97.54 MB	-0% (-144 B) 👌

libdatadog-x64-windows

Artifact	Baseline	Commit	Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll	25.58 MB	25.58 MB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib	89.18 KB	89.18 KB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb	185.54 MB	185.54 MB	0% (0 B) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib	958.60 MB	958.60 MB	-0% (-1016 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll	8.37 MB	8.37 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib	89.18 KB	89.18 KB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb	24.76 MB	24.76 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib	49.28 MB	49.28 MB	-0% (-32 B) 👌

libdatadog-x86-windows

Artifact	Baseline	Commit	Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll	22.23 MB	22.23 MB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib	90.58 KB	90.58 KB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb	190.01 MB	190.00 MB	-0% (-16.00 KB) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib	946.54 MB	946.54 MB	-0% (-994 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll	6.47 MB	6.47 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib	90.58 KB	90.58 KB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb	26.58 MB	26.58 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib	46.89 MB	46.89 MB	-0% (-4 B) 👌

x86_64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a	77.08 MB	77.08 MB	-0% (-40 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so	8.83 MB	8.83 MB	0% (0 B) 👌

x86_64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a	92.50 MB	92.50 MB	-0% (-16 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.74 MB	10.74 MB	0% (0 B) 👌

taegyunkim changed the title ~~profiling: reduce ProfilesDictionary string shard footprint~~ perf(profiling): reduce ProfilesDictionary string shard footprint Jul 16, 2026

taegyunkim force-pushed the taegyun/reduce-profile-string-shards branch from db3ba31 to a7fd92d Compare July 16, 2026 19:15

taegyunkim force-pushed the taegyun/reduce-profile-string-shards branch from a7fd92d to 2a80270 Compare July 16, 2026 19:19

taegyunkim mentioned this pull request Jul 16, 2026

chore(profiling): use libdatadog dictionary arena reduction DataDog/dd-trace-py#19126

Draft

taegyunkim force-pushed the taegyun/reduce-profile-string-shards branch from 2a80270 to d9e28af Compare July 16, 2026 21:12

Reduce profiling dictionary arena footprint

6b4ba8c

taegyunkim force-pushed the taegyun/reduce-profile-string-shards branch from d9e28af to 6b4ba8c Compare July 16, 2026 21:12

taegyunkim changed the title ~~perf(profiling): reduce ProfilesDictionary string shard footprint~~ perf(profiling): reduce ProfilesDictionary arena footprint Jul 16, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf(profiling): reduce ProfilesDictionary arena footprint#2246

perf(profiling): reduce ProfilesDictionary arena footprint#2246
taegyunkim wants to merge 1 commit into
mainfrom
taegyun/reduce-profile-string-shards

taegyunkim commented Jul 16, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 16, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 16, 2026 •

edited

Loading

Uh oh!

pr-commenter Bot commented Jul 16, 2026 •

edited

Loading

Explanation

More details about the CI and significant changes

Group 1

Group 2

Group 3

Group 4

Group 5

Group 6

Group 1

Group 2

Group 3

Group 4

Group 5

Group 6

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Jul 16, 2026 •

edited by datadog-datadog-us1-prod Bot

Loading

Uh oh!

dd-octo-sts Bot commented Jul 16, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

taegyunkim commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Additional Notes

How to test the change?

Uh oh!

github-actions Bot commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📚 Documentation Check Results

📦 libdd-profiling - 650 warning(s)

Uh oh!

github-actions Bot commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔒 Cargo Deny Results

📦 libdd-profiling - 4 error(s)

Uh oh!

pr-commenter Bot commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Comparison

Explanation

More details about the CI and significant changes

scenario:profiles_dictionary/profile_string_inserts/threads/1

scenario:profiles_dictionary/profile_string_inserts/threads/16

scenario:profiles_dictionary/profile_string_inserts/threads/2

scenario:profiles_dictionary/profile_string_inserts/threads/4

Candidate

Group 1

Group 2

Group 3

Group 4

Group 5

Group 6

Baseline

Group 1

Group 2

Group 3

Group 4

Group 5

Group 6

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Jul 16, 2026 • edited by datadog-datadog-us1-prod Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dd-octo-sts Bot commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Artifact Size Benchmark Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

taegyunkim commented Jul 16, 2026 •

edited

Loading

github-actions Bot commented Jul 16, 2026 •

edited

Loading

📦 `libdd-profiling` - 650 warning(s)

github-actions Bot commented Jul 16, 2026 •

edited

Loading

📦 `libdd-profiling` - 4 error(s)

pr-commenter Bot commented Jul 16, 2026 •

edited

Loading

datadog-datadog-prod-us1-2 Bot commented Jul 16, 2026 •

edited by datadog-datadog-us1-prod Bot

Loading

dd-octo-sts Bot commented Jul 16, 2026 •

edited

Loading