Add a whitelist of AWS machines on the benchmark page - #1868
Merged
Conversation
Only the nine standard EC2 instance types are shown in the "Machine" selector; results on any other instance type (m9g, m9gd, x8g, p5, c5) are hidden, so that the set of compared machines stays consistent. Names that are not EC2 instance types (cloud service sizes such as `64GiB` or `Motherduck: jumbo`) are not affected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Member
Author
|
Keep p5.4xlarge as it is the only notable demo for Sirius. |
It is the only notable demo for Sirius. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Member
Author
|
Kept |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why: the benchmark automation was invoked to test some new machine types, but as long as not every system is tested on them, the results should not be in competition.
The "Machine" selector on the benchmark page now shows a fixed whitelist of nine AWS instance types:
t3a.small,c6a.large,c6a.xlarge,c6a.2xlarge,c6a.4xlarge,c8g.4xlarge,c6a.metal,c7a.metal-48xl,c8g.metal-48xlResults on any other EC2 instance type are hidden — currently
m9g.xlarge,m9g.2xlarge,m9gd.xlarge,m9gd.2xlarge,x8g.xlarge,x8g.2xlarge(ClickHouse),p5.4xlarge(Sirius) andc5.4xlarge(VeloDB, from 2022).Machine names that are not EC2 instance types (cloud service sizes such as
64GiB,ClickHouse ☁️: 120GiBorMotherduck: jumbo) are matched by a regexp and left untouched — 38 of them remain.No system disappears from the page: VeloDB still has four other runs, and Sirius still has
lambda-GH200.🤖 Generated with Claude Code