diff --git a/ravel/.gitignore b/ravel/.gitignore new file mode 100644 index 0000000000..f2217dbb5c --- /dev/null +++ b/ravel/.gitignore @@ -0,0 +1,10 @@ +# Artefacts fetched or built by ./install, and run output. +bin/ +src/ +ravel-commit.txt +server.log +server.log.prev +server.pid +*.log +hits.parquet +cache/ diff --git a/ravel/README.md b/ravel/README.md new file mode 100644 index 0000000000..c79ae8d62e --- /dev/null +++ b/ravel/README.md @@ -0,0 +1,162 @@ +# Ravel + +[Ravel](https://github.com/NOFireAI/ravel) is an object-storage-native telemetry +database. Logs, metrics and traces are ingested into immutable objects in +S3-compatible storage, which is the only durable backend: there is no local-disk +storage mode and no local state a restart depends on. ClickBench's `hits` table +is loaded as the logs signal, with each column declared as a typed attribute +column, and queried over SQL. + +## What this entry needs + +Unlike the other S3-using entries in this repository, which read the public +`hits` dataset in place, Ravel *writes* its own objects, so it needs a bucket it +can read and write. + +- The bucket needs no preparation. `./install` qualifies it once, and the first + `./start` bootstraps its `sys/tenancy` marker to the unkeyed (v1) tenant-hash + derivation. The server's own default is the keyed derivation, which refuses to + start without a deployment key file; this benchmark holds no secret and needs + a derivable prefix for `./data-size`, so `./start` pins it unkeyed. +- `RAVEL_S3_BUCKET` (required): a bucket dedicated to this benchmark. Everything + is written under a `t//` prefix, so a shared bucket is correct, + but it is not a fair one: the server's startup cache warmup and its background + catalog fold cover every tenant in the bucket, which spends memory during the + load and CPU beside the timed statements. Measure in a bucket that holds + nothing else. +- `RAVEL_S3_REGION` (default `us-east-1`). +- `RAVEL_CACHE_DIR` (default empty, meaning off): path for the read cache's + local-disk tier. Off by default because on the reference machine it is + slower than the store it caches. A `c6a.4xlarge` has no instance store, so + its disk is a 500 GB gp2 volume measuring 286 MB/s sequential, while this + server sustains 855 MB/s from S3 and peaks at 1,117 MB/s. Set it only where + the disk is genuinely faster: an instance-store box (NVMe at multiple GB/s) + or gp3 with provisioned throughput. +- Credentials: **none are configured**. The server and the CLI run with + `--s3-auth instance-role` and fetch short-lived credentials from IMDSv2, so + the VM's instance profile must allow, for that bucket: + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { "Effect": "Allow", + "Action": ["s3:ListBucket", "s3:GetBucketLocation"], + "Resource": "arn:aws:s3:::YOUR_BUCKET" }, + { "Effect": "Allow", + "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"], + "Resource": "arn:aws:s3:::YOUR_BUCKET/*" } + ] + } + ``` + + Setting any inline credential (`RAVEL_S3_ACCESS_KEY` and friends) alongside + the instance role is refused at startup rather than resolved by precedence, + and `ravel-env.sh` scrubs those variables so an interactive shell's exports + cannot silently turn the run into a static-key run. + +Optional: `RAVEL_TENANT` (default `clickbench`), `RAVEL_SHARDS` (default 4, +must match between `./load` and `./start`), `RAVEL_VERSION` (default the +released version `./install` downloads), `RAVEL_REF` (build that ref from +source instead). + +Run it the usual way: + +```sh +RAVEL_S3_BUCKET=your-bucket RAVEL_S3_REGION=us-east-1 ./benchmark.sh +``` + +## How the run is configured + +`./install` downloads the released `ravel-server` and `ravel-cli` for the host +architecture and verifies them against the release's `SHA256SUMS`; those are the +binaries extracted from the signed container images, so what runs here is what +the published image runs. If the release is unreachable it falls back to +building the pinned ref from source with `--release`. It then runs +`ravel-cli store qualify` once, which is the bucket's one-time conformance +check; the server refuses to start against a store that has never been +qualified. Qualification is setup for the bucket rather than dataset work, so it +sits outside the measured load window. + +`./start` passes **no performance flags**. Since 0.13.0 the server resolves its +query budgets at startup: fetch concurrency from the core count, the read caches +and the two SQL memory ceilings from usable memory (`MemTotal`, capped by the +cgroup memory limit when it runs in a container), with a fixed segment cap and +engine deadline. Every resolved value and its source is logged on a +`performance default resolved` line in `server.log`; a published result should +record those lines, because they are the configuration the numbers were measured +at. + +### The tuned entry + +The `.tuned.json` result is the same harness with four server flags +passed through `RAVEL_TUNED_ARGS` in `./start`: + +``` +--logs-fetch-policy latency-first --fetch-concurrency 256 --sql-max-query-bytes 12884901888 --cache-max-bytes 26323035750 +``` + +`latency-first` is a named policy, not a tuning constant: it says spend +requests to save wall time, and resolves the byte quantities exactly as +`byte-minimal` does, so a logs read takes ranged reads wherever they save +bytes. It only pays off once fetch concurrency is raised with it, which is what +the second flag does (it sets the object-store GET permits, the SQL partition +count and the PromQL fan-out together). The third flag lifts the per-query +memory pool from the value derived on a 30 GB machine (about 8.2 GB) to 12 GiB, +which is what lets the widest `GROUP BY` in the set (q33) complete instead of +being refused. The trade this entry accepts is more object-store requests for +less cold wall-clock; on the reference corpus it moves about 5.3x the GET +requests of the stock entry. + +The fourth flag sets the read cache to 26.3 GB (80% of this machine's memory) +so the whole 11.7 GB dataset can stay resident between the three runs of a +query. Measured, it changes the tuned entry's totals by about 1%: with +`latency-first` a query reads ranged blocks rather than whole objects (q2 +moves 1.9 GB instead of 11.7 GB), and that working set already fits the +default 8.2 GB cache, so the tuned warm runs were served from cache before the +flag. It is kept because it makes the entry's cache behaviour independent of +the dataset size rather than incidental to it. The stock entry is where the +cache size decides the result: its derived cache is 25% of memory (8.2 GB), +sized under a ten-connection load test where a larger cache left too little +headroom for query demand, and whole-object reads of this dataset do not fit +in it, so the stock warm runs re-fetch every object. ClickBench runs one query +at a time, which is not that concurrent window, so the larger cache is safe +here and is presented as what it is: a tuning for this benchmark's protocol, +not a recommended default. Measured on this machine, one server left running +across the whole set at this cache size is out-of-memory killed at q33 once +the cache has filled from the earlier queries; the protocol's restart before +every query is what makes the setting safe here. Reproduce it with: + +``` +RAVEL_TUNED_ARGS="--logs-fetch-policy latency-first --fetch-concurrency 256 --sql-max-query-bytes 12884901888 --cache-max-bytes 26323035750" ./benchmark.sh +``` + +`./load` declares the typed attribute columns and then loads the Parquet file. +Object size is set at ingest by `--batch-rows`: one batch becomes one object per +involved shard, so 150,000 rows over 4 shards gives ~4 MB objects and roughly +2,600 objects for the 100M-row dataset. **There is no post-load step** — no +compaction, no catalog fold, no VACUUM equivalent — so the layout the queries +run against is the layout ingest produced. + +`./data-size` reports the tenant's whole durable footprint in the bucket (data +objects, commit records, manifests, catalog snapshots), which is where all of +Ravel's state lives. The local disk holds only the downloaded Parquet file and +the read cache. + +## Notes + +- Queries go over the HTTP SQL endpoint on loopback, one process per query, the + same shape as the other daemon entries. +- Ravel is a telemetry database rather than a general-purpose warehouse: the + `hits` columns are modelled as typed attribute columns on log records, and + every query states a time window covering the dataset because the SQL + endpoint's window defaults to the last hour. +- The heaviest whole-table aggregates can exceed the derived per-query memory + pool on a small instance and are reported as errors rather than being run + with a raised limit. +- The stock entry's warm runs are close to its cold runs. Ravel holds no data + on local disk, so a warm run is served from the read cache or not at all, + and the stock cache (25% of memory, 8.2 GB here) is smaller than this + dataset (11.7 GB). Every warm run therefore re-reads the objects it needs + from object storage. The tuned entry shows the same binary with a cache the + dataset fits in. diff --git a/ravel/benchmark.sh b/ravel/benchmark.sh new file mode 100755 index 0000000000..a20f8f8e25 --- /dev/null +++ b/ravel/benchmark.sh @@ -0,0 +1,19 @@ +#!/bin/bash +# Ravel: object-storage-native telemetry database, queried over SQL. +# +# Ravel keeps every durable byte in S3-compatible object storage; there is no +# local-disk storage mode. The bucket is supplied by the operator through +# RAVEL_S3_BUCKET and the credentials come from the EC2 instance profile, so no +# key is stored in this repository. See README.md for the required permissions. +export BENCH_DOWNLOAD_SCRIPT="download-hits-parquet-single" + +# The server is a daemon whose data survives a restart (it is in object +# storage), so the driver's defaults are right: restartable, durable, and the +# concurrent-QPS test applies. +# +# First start has to reach the object store, resolve the tenant and open the +# catalog, which is slower than a local-disk engine's start; give ./check room +# rather than failing a run on a cold control-plane round trip. +export BENCH_CHECK_TIMEOUT="${BENCH_CHECK_TIMEOUT:-600}" + +exec ../lib/benchmark-common.sh diff --git a/ravel/check b/ravel/check new file mode 100755 index 0000000000..f7f209eb3d --- /dev/null +++ b/ravel/check @@ -0,0 +1,53 @@ +#!/bin/bash +# Succeeds only when the server this harness started is the one answering. +# +# The driver (lib/benchmark-common.sh) treats ./check as the authoritative +# readiness signal and tolerates non-zero exits from ./start and ./stop. So a +# stale server left running by a failed ./stop would, with a plain HTTP probe, +# make the next "cold" try pass warm and nothing would notice. This check +# therefore requires all of: the pidfile ./start wrote exists, that pid is +# alive and is a ravel-server, that pid is the process listening on the HTTP +# port, and SELECT 1 succeeds against it. A server answering on the port that +# this harness did not launch fails the check, and the driver's check loop then +# times out loudly instead of measuring the wrong process. +# +# `ss -p` shows the listening pid only for the same user or root. The +# benchmark runs as root; run as another user the owner is invisible and this +# fails closed with the message below rather than passing. +set -eu +. ./ravel-env.sh + +pidfile="$RAVEL_PIDFILE" +if [ ! -f "$pidfile" ]; then + echo "check: no pidfile at $pidfile; nothing this harness started is running" >&2 + exit 1 +fi +pid=$(cat "$pidfile" 2>/dev/null || true) +case "$pid" in + ''|*[!0-9]*) echo "check: $pidfile holds no pid" >&2; exit 1 ;; +esac +if ! kill -0 "$pid" 2>/dev/null; then + echo "check: pid $pid from $pidfile is not running" >&2 + exit 1 +fi +if ! grep -qs 'ravel-server' "/proc/$pid/cmdline" 2>/dev/null; then + echo "check: pid $pid is not a ravel-server" >&2 + exit 1 +fi + +port="${RAVEL_HTTP##*:}" +owner=$(ss -ltnpH "( sport = :$port )" 2>/dev/null | sed -n 's/.*pid=\([0-9][0-9]*\).*/\1/p' | head -n 1) +if [ -z "$owner" ]; then + echo "check: no visible pid is listening on :$port (needs root or the same user)" >&2 + exit 1 +fi +if [ "$owner" != "$pid" ]; then + echo "check: :$port is held by pid $owner, not the pid $pid this harness started" >&2 + exit 1 +fi + +curl -sSf -X POST \ + -H 'Content-Type: application/json' \ + -H "x-ravel-tenant: $RAVEL_TENANT" \ + --data-binary '{"query":"SELECT 1","start":0,"end":4102444800}' \ + "http://${RAVEL_HTTP}/api/v1/sql" >/dev/null diff --git a/ravel/data-size b/ravel/data-size new file mode 100755 index 0000000000..d76016af4f --- /dev/null +++ b/ravel/data-size @@ -0,0 +1,43 @@ +#!/bin/bash +# Ravel keeps every durable byte in object storage, so the VM's local disk +# footprint is not the data size. This sums the bytes stored under this +# tenant's own prefix in the bucket: data objects, commit records, manifests +# and catalog snapshots, which is the tenant's whole durable footprint. +set -eu +. ./ravel-env.sh + +# The tenant id is hashed under the bucket's pinned scheme, so the prefix is not +# derivable from the id and no command prints it (NOFireAI/ravel#1180). Take it +# from the key `catalog inspect` reports it addressed, which it names whether or +# not a catalog HEAD has been published yet. +hash=$("$RAVEL_CLI" --store s3 --s3-auth instance-role catalog inspect \ + --tenant "$RAVEL_TENANT" --signal logs 2>&1 \ + | grep -oE 't/[0-9a-f]+/' | head -1 | cut -d/ -f2) + +# Fallback: a bucket dedicated to this benchmark holds one tenant, so its single +# prefix is unambiguous. Anything else is ambiguous and gets no guess. +if [ -z "$hash" ]; then + prefixes=$(aws s3 ls "s3://${RAVEL_S3_BUCKET}/t/" | awk '$1 == "PRE" { print $2 }' | tr -d /) + count=$(printf '%s\n' "$prefixes" | awk 'NF' | wc -l | tr -d ' ') + if [ "$count" = 1 ]; then + hash=$(printf '%s\n' "$prefixes" | awk 'NF') + echo "data-size: resolved the prefix as the bucket's only tenant" >&2 + fi +fi + +if [ -z "$hash" ]; then + echo "data-size: could not resolve the object prefix for tenant '$RAVEL_TENANT'" >&2 + exit 1 +fi + +bytes=$(aws s3 ls --summarize --recursive "s3://${RAVEL_S3_BUCKET}/t/${hash}/" \ + | awk '/Total Size:/ { print $3 }') + +# A tenant that measured zero means the prefix resolved to the wrong place, or +# the load did not land. Either way it is not a data size. +if [ -z "$bytes" ] || [ "$bytes" -eq 0 ]; then + echo "data-size: prefix t/${hash}/ holds no objects" >&2 + exit 1 +fi + +echo "$bytes" diff --git a/ravel/hits.mapping.toml b/ravel/hits.mapping.toml new file mode 100644 index 0000000000..9ebaf15486 --- /dev/null +++ b/ravel/hits.mapping.toml @@ -0,0 +1,586 @@ +# ClickBench `hits` schema mapping for `ravel-cli load --parquet` (ADR-0100, issue #430). +# +# Source of truth for the column names and their upstream SQL types: +# https://github.com/ClickHouse/ClickBench/blob/main/postgresql/create.sql +# (105 columns). This mapping is checked in so a ClickBench load is repeatable; +# it is consumed twice, with the SAME key/type information: +# 1. `ravel-cli load --parquet --mapping hits.mapping.toml` writes the records. +# 2. `ravel-cli typed-attr-column set --from-mapping hits.mapping.toml` declares +# the typed attribute columns the SQL layer accelerates (ADR-0100 decision 2). +# +# Type mapping (upstream SQL type -> loader ColType -> ravel-sql DeclaredType): +# BIGINT / INTEGER / SMALLINT -> i64 -> I64 +# TEXT / VARCHAR(n) / CHAR -> str -> Str +# Date / TIMESTAMP -> i64 -> I64 (see EventTime / secondary-time note) +# +# EventTime is the primary event-time column: it rides the native typed `ts` +# path (ts_column below) and is NOT declared as an attribute. Every OTHER column +# becomes a typed attribute with its ColType. +# +# F64 note (ADR-0101 / #431): ColType::F64 attributes CANNOT be declared as typed +# columns -- ravel-sql `DeclaredType` has no F64 variant (Str/I64/Bool/Bytes only), +# so `typed-attr-column set --from-mapping` skips an f64 entry with a stderr warning +# and a float attribute stays queryable only through `attrs['']`. This mapping +# declares NO f64 column: the `hits` schema is entirely integer, string, and +# date/time columns (77 integer/date/time, 28 text; 0 float), so the F64 gap affects +# 0 of the 105 columns here. The note is kept because a different dataset would hit it. +# +# Secondary time columns EventDate, ClientEventTime, LocalEventTime are declared i64. +# EventDate is a Date column: the loader stores it as its native unit, epoch DAYS +# (Date32 = days since 1970-01-01), NOT nanoseconds (load.rs read_i64). A date-literal +# comparison in a query therefore becomes an epoch-DAY integer comparison; the corpus +# flags those statements `modified` (ADR-0100 decision 3). ClientEventTime and +# LocalEventTime are TIMESTAMP in the schema: the loader `i64` path accepts integer +# and Date32/Date64 columns but NOT a native Arrow Timestamp column, so these two load +# only if the operator's hits.parquet encodes them as an integer/date column. None of +# the 43 ClickBench statements reference them, so this does not affect the suite. + +ts_column = "EventTime" +# EventTime in the original data is a Unix-seconds datetime. If the operator's +# hits.parquet stores EventTime as a native Arrow Timestamp, the loader uses the +# column's own unit and this ts_unit is ignored (load.rs read_ts); set it to match +# the parquet only when EventTime is stored as an integer. +ts_unit = "seconds" + +# CounterID (issue #519): the one column in `hits` that acts as a per-entity +# key -- ClickBench's "counter" is the site/tag being measured, so CounterID +# is the natural stream boundary. Declared as a resource_attribute rather than +# an attribute so it becomes part of stream identity (ADR-0029 log_stream_id): +# `shard_for_log` hashes resource attributes, so this is what actually spreads +# ingest across `--shards`. In the pre-#519 mapping, CounterID was a record +# attribute: every row's resource-attribute set was empty, so every row hashed +# to the same stream and 100% of writes landed on one shard regardless of +# --shards -- one core did all merge+encode work no matter how many shards +# were provisioned. +# +# `ravel-cli load` runs with a fixed `target_bytes: 1` (one Strict flush per +# batch), so raising --shards without raising --batch-rows to compensate +# produces MORE, SMALLER objects: a batch's rows now split across shards by +# CounterID hash, and each shard's slice flushes immediately as its own +# object, before it can reach the `block_target_records` target (8192). See +# docs/guides/clickbench.md step 2 for the --batch-rows this requires. +# +# The other 103 non-EventTime columns stay record attributes (typed values in +# `attrs`, not part of stream identity). 104 declared columns total (103 +# attribute + 1 resource_attribute) sit far under the per-object dynamic-column +# budget of 1000 (RlogConfig::max_dynamic_columns). + +[[resource_attribute]] +key = "CounterID" +column = "CounterID" +type = "i64" + +[[attribute]] +key = "WatchID" +column = "WatchID" +type = "i64" + +[[attribute]] +key = "JavaEnable" +column = "JavaEnable" +type = "i64" + +[[attribute]] +key = "Title" +column = "Title" +type = "str" + +[[attribute]] +key = "GoodEvent" +column = "GoodEvent" +type = "i64" + +[[attribute]] +key = "EventDate" +column = "EventDate" +type = "i64" + +[[attribute]] +key = "ClientIP" +column = "ClientIP" +type = "i64" + +[[attribute]] +key = "RegionID" +column = "RegionID" +type = "i64" + +[[attribute]] +key = "UserID" +column = "UserID" +type = "i64" + +[[attribute]] +key = "CounterClass" +column = "CounterClass" +type = "i64" + +[[attribute]] +key = "OS" +column = "OS" +type = "i64" + +[[attribute]] +key = "UserAgent" +column = "UserAgent" +type = "i64" + +[[attribute]] +key = "URL" +column = "URL" +type = "str" + +[[attribute]] +key = "Referer" +column = "Referer" +type = "str" + +[[attribute]] +key = "IsRefresh" +column = "IsRefresh" +type = "i64" + +[[attribute]] +key = "RefererCategoryID" +column = "RefererCategoryID" +type = "i64" + +[[attribute]] +key = "RefererRegionID" +column = "RefererRegionID" +type = "i64" + +[[attribute]] +key = "URLCategoryID" +column = "URLCategoryID" +type = "i64" + +[[attribute]] +key = "URLRegionID" +column = "URLRegionID" +type = "i64" + +[[attribute]] +key = "ResolutionWidth" +column = "ResolutionWidth" +type = "i64" + +[[attribute]] +key = "ResolutionHeight" +column = "ResolutionHeight" +type = "i64" + +[[attribute]] +key = "ResolutionDepth" +column = "ResolutionDepth" +type = "i64" + +[[attribute]] +key = "FlashMajor" +column = "FlashMajor" +type = "i64" + +[[attribute]] +key = "FlashMinor" +column = "FlashMinor" +type = "i64" + +[[attribute]] +key = "FlashMinor2" +column = "FlashMinor2" +type = "str" + +[[attribute]] +key = "NetMajor" +column = "NetMajor" +type = "i64" + +[[attribute]] +key = "NetMinor" +column = "NetMinor" +type = "i64" + +[[attribute]] +key = "UserAgentMajor" +column = "UserAgentMajor" +type = "i64" + +[[attribute]] +key = "UserAgentMinor" +column = "UserAgentMinor" +type = "str" + +[[attribute]] +key = "CookieEnable" +column = "CookieEnable" +type = "i64" + +[[attribute]] +key = "JavascriptEnable" +column = "JavascriptEnable" +type = "i64" + +[[attribute]] +key = "IsMobile" +column = "IsMobile" +type = "i64" + +[[attribute]] +key = "MobilePhone" +column = "MobilePhone" +type = "i64" + +[[attribute]] +key = "MobilePhoneModel" +column = "MobilePhoneModel" +type = "str" + +[[attribute]] +key = "Params" +column = "Params" +type = "str" + +[[attribute]] +key = "IPNetworkID" +column = "IPNetworkID" +type = "i64" + +[[attribute]] +key = "TraficSourceID" +column = "TraficSourceID" +type = "i64" + +[[attribute]] +key = "SearchEngineID" +column = "SearchEngineID" +type = "i64" + +[[attribute]] +key = "SearchPhrase" +column = "SearchPhrase" +type = "str" + +[[attribute]] +key = "AdvEngineID" +column = "AdvEngineID" +type = "i64" + +[[attribute]] +key = "IsArtifical" +column = "IsArtifical" +type = "i64" + +[[attribute]] +key = "WindowClientWidth" +column = "WindowClientWidth" +type = "i64" + +[[attribute]] +key = "WindowClientHeight" +column = "WindowClientHeight" +type = "i64" + +[[attribute]] +key = "ClientTimeZone" +column = "ClientTimeZone" +type = "i64" + +[[attribute]] +key = "ClientEventTime" +column = "ClientEventTime" +type = "i64" + +[[attribute]] +key = "SilverlightVersion1" +column = "SilverlightVersion1" +type = "i64" + +[[attribute]] +key = "SilverlightVersion2" +column = "SilverlightVersion2" +type = "i64" + +[[attribute]] +key = "SilverlightVersion3" +column = "SilverlightVersion3" +type = "i64" + +[[attribute]] +key = "SilverlightVersion4" +column = "SilverlightVersion4" +type = "i64" + +[[attribute]] +key = "PageCharset" +column = "PageCharset" +type = "str" + +[[attribute]] +key = "CodeVersion" +column = "CodeVersion" +type = "i64" + +[[attribute]] +key = "IsLink" +column = "IsLink" +type = "i64" + +[[attribute]] +key = "IsDownload" +column = "IsDownload" +type = "i64" + +[[attribute]] +key = "IsNotBounce" +column = "IsNotBounce" +type = "i64" + +[[attribute]] +key = "FUniqID" +column = "FUniqID" +type = "i64" + +[[attribute]] +key = "OriginalURL" +column = "OriginalURL" +type = "str" + +[[attribute]] +key = "HID" +column = "HID" +type = "i64" + +[[attribute]] +key = "IsOldCounter" +column = "IsOldCounter" +type = "i64" + +[[attribute]] +key = "IsEvent" +column = "IsEvent" +type = "i64" + +[[attribute]] +key = "IsParameter" +column = "IsParameter" +type = "i64" + +[[attribute]] +key = "DontCountHits" +column = "DontCountHits" +type = "i64" + +[[attribute]] +key = "WithHash" +column = "WithHash" +type = "i64" + +[[attribute]] +key = "HitColor" +column = "HitColor" +type = "str" + +[[attribute]] +key = "LocalEventTime" +column = "LocalEventTime" +type = "i64" + +[[attribute]] +key = "Age" +column = "Age" +type = "i64" + +[[attribute]] +key = "Sex" +column = "Sex" +type = "i64" + +[[attribute]] +key = "Income" +column = "Income" +type = "i64" + +[[attribute]] +key = "Interests" +column = "Interests" +type = "i64" + +[[attribute]] +key = "Robotness" +column = "Robotness" +type = "i64" + +[[attribute]] +key = "RemoteIP" +column = "RemoteIP" +type = "i64" + +[[attribute]] +key = "WindowName" +column = "WindowName" +type = "i64" + +[[attribute]] +key = "OpenerName" +column = "OpenerName" +type = "i64" + +[[attribute]] +key = "HistoryLength" +column = "HistoryLength" +type = "i64" + +[[attribute]] +key = "BrowserLanguage" +column = "BrowserLanguage" +type = "str" + +[[attribute]] +key = "BrowserCountry" +column = "BrowserCountry" +type = "str" + +[[attribute]] +key = "SocialNetwork" +column = "SocialNetwork" +type = "str" + +[[attribute]] +key = "SocialAction" +column = "SocialAction" +type = "str" + +[[attribute]] +key = "HTTPError" +column = "HTTPError" +type = "i64" + +[[attribute]] +key = "SendTiming" +column = "SendTiming" +type = "i64" + +[[attribute]] +key = "DNSTiming" +column = "DNSTiming" +type = "i64" + +[[attribute]] +key = "ConnectTiming" +column = "ConnectTiming" +type = "i64" + +[[attribute]] +key = "ResponseStartTiming" +column = "ResponseStartTiming" +type = "i64" + +[[attribute]] +key = "ResponseEndTiming" +column = "ResponseEndTiming" +type = "i64" + +[[attribute]] +key = "FetchTiming" +column = "FetchTiming" +type = "i64" + +[[attribute]] +key = "SocialSourceNetworkID" +column = "SocialSourceNetworkID" +type = "i64" + +[[attribute]] +key = "SocialSourcePage" +column = "SocialSourcePage" +type = "str" + +[[attribute]] +key = "ParamPrice" +column = "ParamPrice" +type = "i64" + +[[attribute]] +key = "ParamOrderID" +column = "ParamOrderID" +type = "str" + +[[attribute]] +key = "ParamCurrency" +column = "ParamCurrency" +type = "str" + +[[attribute]] +key = "ParamCurrencyID" +column = "ParamCurrencyID" +type = "i64" + +[[attribute]] +key = "OpenstatServiceName" +column = "OpenstatServiceName" +type = "str" + +[[attribute]] +key = "OpenstatCampaignID" +column = "OpenstatCampaignID" +type = "str" + +[[attribute]] +key = "OpenstatAdID" +column = "OpenstatAdID" +type = "str" + +[[attribute]] +key = "OpenstatSourceID" +column = "OpenstatSourceID" +type = "str" + +[[attribute]] +key = "UTMSource" +column = "UTMSource" +type = "str" + +[[attribute]] +key = "UTMMedium" +column = "UTMMedium" +type = "str" + +[[attribute]] +key = "UTMCampaign" +column = "UTMCampaign" +type = "str" + +[[attribute]] +key = "UTMContent" +column = "UTMContent" +type = "str" + +[[attribute]] +key = "UTMTerm" +column = "UTMTerm" +type = "str" + +[[attribute]] +key = "FromTag" +column = "FromTag" +type = "str" + +[[attribute]] +key = "HasGCLID" +column = "HasGCLID" +type = "i64" + +[[attribute]] +key = "RefererHash" +column = "RefererHash" +type = "i64" + +[[attribute]] +key = "URLHash" +column = "URLHash" +type = "i64" + +[[attribute]] +key = "CLID" +column = "CLID" +type = "i64" diff --git a/ravel/install b/ravel/install new file mode 100755 index 0000000000..026af9ca75 --- /dev/null +++ b/ravel/install @@ -0,0 +1,109 @@ +#!/bin/bash +set -eu +. ./ravel-env.sh + +# A server left over from a previous run holds the binaries this script is about +# to replace, and ./start is idempotent through ./check, so a survivor would go +# on serving and the run would measure the old build. +./stop + +# ClickBench standardises on Ubuntu, so apt is the primary path; the dnf branch +# keeps the entry runnable on the Amazon Linux images some of these VMs ship +# with rather than failing at the first line. +if command -v apt-get >/dev/null 2>&1; then + sudo apt-get update -qq + # awscli is deliberately not in this list: Ubuntu 24.04 ships no awscli + # package ("has no installation candidate"), so it is installed from + # Amazon's own archive below on every distribution. + sudo apt-get install -y --no-install-recommends jq curl ca-certificates unzip +elif command -v dnf >/dev/null 2>&1; then + # curl is deliberately absent: Amazon Linux ships curl-minimal, which + # provides the binary and conflicts with the full curl package. + sudo dnf install -y -q jq ca-certificates unzip +else + echo "install: no supported package manager (apt-get or dnf)" >&2 + exit 1 +fi + +# ./data-size sums the tenant's bytes in the bucket, which needs the AWS CLI. +# Installed from Amazon's archive rather than from a distribution package so +# the entry does not depend on which distributions still carry one. +if ! command -v aws >/dev/null 2>&1; then + case "$(uname -m)" in + x86_64|amd64) awsarch=x86_64 ;; + aarch64|arm64) awsarch=aarch64 ;; + *) echo "install: unsupported arch $(uname -m) for the AWS CLI" >&2; exit 1 ;; + esac + curl -fsSL --retry 3 "https://awscli.amazonaws.com/awscli-exe-linux-${awsarch}.zip" -o /tmp/awscliv2.zip + rm -rf /tmp/aws + unzip -q /tmp/awscliv2.zip -d /tmp + sudo /tmp/aws/install --update +fi +aws --version + +RAVEL_VERSION="${RAVEL_VERSION:-v0.14.0}" +RAVEL_REPO="${RAVEL_REPO:-https://github.com/NOFireAI/ravel}" + +case "$(uname -m)" in + x86_64|amd64) arch=amd64 ;; + aarch64|arm64) arch=arm64 ;; + *) echo "install: unsupported arch $(uname -m)" >&2; exit 1 ;; +esac + +mkdir -p "$RAVEL_BIN_DIR" + +# Preferred path: the released binaries, which are the ones extracted from the +# signed container images, so a downloaded binary is byte-identical to what the +# published image runs (ADR-0086). Checksums are verified against the release's +# own SHA256SUMS. +base="${RAVEL_REPO}/releases/download/${RAVEL_VERSION}" +if curl -fsSL --retry 3 -o "$RAVEL_BIN_DIR/SHA256SUMS" "${base}/SHA256SUMS"; then + # Downloaded under a temporary name and moved into place only once the + # checksum passes, so an unverified binary never sits at the path ./start + # runs. The rename also replaces a binary that is still executing, which + # writing to it in place does not: curl fails with ETXTBSY. + for b in ravel-server ravel-cli; do + curl -fsSL --retry 3 -o "$RAVEL_BIN_DIR/${b}.new" "${base}/${b}-linux-${arch}" + done + ( cd "$RAVEL_BIN_DIR" && for b in ravel-server ravel-cli; do + want=$(awk -v n="${b}-linux-${arch}" '$2 == n || $2 == "*"n {print $1}' SHA256SUMS | head -1) + [ -n "$want" ] || { echo "install: ${b}-linux-${arch} absent from SHA256SUMS" >&2; exit 1; } + have=$(sha256sum "${b}.new" | awk '{print $1}') + [ "$want" = "$have" ] || { echo "install: checksum mismatch for $b" >&2; exit 1; } + chmod +x "${b}.new" + mv -f "${b}.new" "$b" + done ) +else + # Fallback: build at a ref. Used when the release is unreachable, or to + # benchmark an unreleased commit via RAVEL_REF. + echo "install: release ${RAVEL_VERSION} not reachable, building from source" >&2 + if command -v apt-get >/dev/null 2>&1; then + sudo apt-get install -y --no-install-recommends git build-essential pkg-config libssl-dev protobuf-compiler + else + sudo dnf install -y -q git gcc gcc-c++ make pkgconfig openssl-devel protobuf-compiler + fi + if ! cargo --version >/dev/null 2>&1; then + if command -v rustup >/dev/null 2>&1; then rustup default stable + else curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal; fi + fi + # shellcheck disable=SC1091 + [ -f "$HOME/.cargo/env" ] && . "$HOME/.cargo/env" + [ -d src ] || git clone --filter=blob:none "${RAVEL_REPO}.git" src + git -C src fetch origin --quiet + git -C src checkout -q --detach "${RAVEL_REF:-$RAVEL_VERSION}" + git -C src rev-parse HEAD > ravel-commit.txt + # The SQL surface sits behind a cargo feature that is off by default. + # --release matters: the gate profile is opt-level 0 and loads ~7x slower. + cargo build --manifest-path src/Cargo.toml --release -p ravel-server --features sql + cargo build --manifest-path src/Cargo.toml --release -p ravel-cli + cp src/target/release/ravel-server src/target/release/ravel-cli "$RAVEL_BIN_DIR/" +fi + +"$RAVEL_SERVER" --version || true +"$RAVEL_CLI" --version || true + +# One-time bucket provisioning (ADR-0050 section 6): the server refuses to start +# against a store that has never been qualified. It lives here rather than in +# ./load because it qualifies the bucket rather than loading the dataset, and +# ./load's wall clock is the reported Load time. +"$RAVEL_CLI" --store s3 --s3-auth instance-role store qualify diff --git a/ravel/load b/ravel/load new file mode 100755 index 0000000000..2f69f5b287 --- /dev/null +++ b/ravel/load @@ -0,0 +1,52 @@ +#!/bin/bash +# Loads hits.parquet into Ravel. The driver times this script and prints the +# "Load time:" line itself, so nothing here prints one. +# +# Everything this writes lands in object storage. There is no post-load step: +# no compaction, no catalog fold, no VACUUM equivalent. Object sizing is done +# at ingest instead, by the geometry below. +set -eu +. ./ravel-env.sh + +PARQUET="${RAVEL_PARQUET:-hits.parquet}" +MAPPING="${RAVEL_MAPPING:-hits.mapping.toml}" +[ -f "$PARQUET" ] || { echo "load: $PARQUET not found (the driver's download step should have fetched it)" >&2; exit 1; } +[ -f "$MAPPING" ] || { echo "load: $MAPPING not found" >&2; exit 1; } + +# `ravel-cli load` drives the ingest router in-process and writes to object +# storage itself, so the server takes no part in the load. A server left running +# through it does no work for the load while its startup cache warmup and its +# background catalog folds compete for the same memory the loader needs. That is +# not a small effect: on a 32 GB box the read cache alone resolves to a 26.3 GB +# ceiling and the kernel OOM-killed a 24.1 GB ravel-server six seconds before the +# last row landed (NOFireAI/ravel#1170). Stop it here, start it again at the end. +./stop + +# Declare the typed attribute columns before the first row lands. The mapping +# names each hits column and its type; declaring them lets ingest stamp exact +# per-column statistics on the commit records, which is what lets aggregates be +# answered without reading data objects. +"$RAVEL_CLI" --store s3 --s3-auth instance-role \ + typed-attr-column set "$RAVEL_TENANT" --from-mapping "$MAPPING" + +# Load geometry. One batch becomes one object per involved shard, so +# --batch-rows sets the object size directly: 150,000 rows over 4 shards is +# ~37.5k rows (~4 MB) per object and ~2,600 objects for the 100M-row dataset. +# That is the whole object-sizing mechanism; --target-bytes stays at its +# default of 1 (flush every write) because accumulating instead makes each +# batch's acknowledgement wait for a later batch and the load runs several +# times longer for the same layout. +"$RAVEL_CLI" --store s3 --s3-auth instance-role load \ + --parquet "$PARQUET" \ + --tenant "$RAVEL_TENANT" \ + --mapping "$MAPPING" \ + --shards "$RAVEL_SHARDS" \ + --read-cursors "${RAVEL_READ_CURSORS:-16}" \ + --batch-rows "${RAVEL_BATCH_ROWS:-150000}" \ + --target-bytes 1 + +sync + +# Serving again for ./check and the query phase. ./start waits for the listener, +# so this returns only once the server answers. +./start diff --git a/ravel/queries.sql b/ravel/queries.sql new file mode 100644 index 0000000000..1dedee62a9 --- /dev/null +++ b/ravel/queries.sql @@ -0,0 +1,43 @@ +SELECT COUNT(*) FROM logs; +SELECT COUNT(*) FROM logs WHERE "AdvEngineID" <> 0; +SELECT SUM("AdvEngineID"), COUNT(*), AVG("ResolutionWidth") FROM logs; +SELECT AVG("UserID") FROM logs; +SELECT COUNT(DISTINCT "UserID") FROM logs; +SELECT COUNT(DISTINCT "SearchPhrase") FROM logs; +SELECT MIN("EventDate"), MAX("EventDate") FROM logs; +SELECT "AdvEngineID", COUNT(*) FROM logs WHERE "AdvEngineID" <> 0 GROUP BY "AdvEngineID" ORDER BY COUNT(*) DESC; +SELECT "RegionID", COUNT(DISTINCT "UserID") AS u FROM logs GROUP BY "RegionID" ORDER BY u DESC LIMIT 10; +SELECT "RegionID", SUM("AdvEngineID"), COUNT(*) AS c, AVG("ResolutionWidth"), COUNT(DISTINCT "UserID") FROM logs GROUP BY "RegionID" ORDER BY c DESC LIMIT 10; +SELECT "MobilePhoneModel", COUNT(DISTINCT "UserID") AS u FROM logs WHERE "MobilePhoneModel" <> '' GROUP BY "MobilePhoneModel" ORDER BY u DESC LIMIT 10; +SELECT "MobilePhone", "MobilePhoneModel", COUNT(DISTINCT "UserID") AS u FROM logs WHERE "MobilePhoneModel" <> '' GROUP BY "MobilePhone", "MobilePhoneModel" ORDER BY u DESC LIMIT 10; +SELECT "SearchPhrase", COUNT(*) AS c FROM logs WHERE "SearchPhrase" <> '' GROUP BY "SearchPhrase" ORDER BY c DESC LIMIT 10; +SELECT "SearchPhrase", COUNT(DISTINCT "UserID") AS u FROM logs WHERE "SearchPhrase" <> '' GROUP BY "SearchPhrase" ORDER BY u DESC LIMIT 10; +SELECT "SearchEngineID", "SearchPhrase", COUNT(*) AS c FROM logs WHERE "SearchPhrase" <> '' GROUP BY "SearchEngineID", "SearchPhrase" ORDER BY c DESC LIMIT 10; +SELECT "UserID", COUNT(*) FROM logs GROUP BY "UserID" ORDER BY COUNT(*) DESC LIMIT 10; +SELECT "UserID", "SearchPhrase", COUNT(*) FROM logs GROUP BY "UserID", "SearchPhrase" ORDER BY COUNT(*) DESC LIMIT 10; +SELECT "UserID", "SearchPhrase", COUNT(*) FROM logs GROUP BY "UserID", "SearchPhrase" LIMIT 10; +SELECT "UserID", date_part('minute', ts) AS m, "SearchPhrase", COUNT(*) FROM logs GROUP BY "UserID", m, "SearchPhrase" ORDER BY COUNT(*) DESC LIMIT 10; +SELECT "UserID" FROM logs WHERE "UserID" = 435090932899640449; +SELECT COUNT(*) FROM logs WHERE "URL" LIKE '%google%'; +SELECT "SearchPhrase", MIN("URL"), COUNT(*) AS c FROM logs WHERE "URL" LIKE '%google%' AND "SearchPhrase" <> '' GROUP BY "SearchPhrase" ORDER BY c DESC LIMIT 10; +SELECT "SearchPhrase", MIN("URL"), MIN("Title"), COUNT(*) AS c, COUNT(DISTINCT "UserID") FROM logs WHERE "Title" LIKE '%Google%' AND "URL" NOT LIKE '%.google.%' AND "SearchPhrase" <> '' GROUP BY "SearchPhrase" ORDER BY c DESC LIMIT 10; +SELECT * FROM logs WHERE "URL" LIKE '%google%' ORDER BY ts LIMIT 10; +SELECT "SearchPhrase" FROM logs WHERE "SearchPhrase" <> '' ORDER BY ts LIMIT 10; +SELECT "SearchPhrase" FROM logs WHERE "SearchPhrase" <> '' ORDER BY "SearchPhrase" LIMIT 10; +SELECT "SearchPhrase" FROM logs WHERE "SearchPhrase" <> '' ORDER BY ts, "SearchPhrase" LIMIT 10; +SELECT "CounterID", AVG(length("URL")) AS l, COUNT(*) AS c FROM logs WHERE "URL" <> '' GROUP BY "CounterID" HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; +SELECT regexp_replace("Referer", '^https?://(?:www\.)?([^/]+)/.*$', '\1') AS k, AVG(length("Referer")) AS l, COUNT(*) AS c, MIN("Referer") FROM logs WHERE "Referer" <> '' GROUP BY k HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; +SELECT SUM("ResolutionWidth"), SUM("ResolutionWidth" + 1), SUM("ResolutionWidth" + 2), SUM("ResolutionWidth" + 3), SUM("ResolutionWidth" + 4), SUM("ResolutionWidth" + 5), SUM("ResolutionWidth" + 6), SUM("ResolutionWidth" + 7), SUM("ResolutionWidth" + 8), SUM("ResolutionWidth" + 9), SUM("ResolutionWidth" + 10), SUM("ResolutionWidth" + 11), SUM("ResolutionWidth" + 12), SUM("ResolutionWidth" + 13), SUM("ResolutionWidth" + 14), SUM("ResolutionWidth" + 15), SUM("ResolutionWidth" + 16), SUM("ResolutionWidth" + 17), SUM("ResolutionWidth" + 18), SUM("ResolutionWidth" + 19), SUM("ResolutionWidth" + 20), SUM("ResolutionWidth" + 21), SUM("ResolutionWidth" + 22), SUM("ResolutionWidth" + 23), SUM("ResolutionWidth" + 24), SUM("ResolutionWidth" + 25), SUM("ResolutionWidth" + 26), SUM("ResolutionWidth" + 27), SUM("ResolutionWidth" + 28), SUM("ResolutionWidth" + 29), SUM("ResolutionWidth" + 30), SUM("ResolutionWidth" + 31), SUM("ResolutionWidth" + 32), SUM("ResolutionWidth" + 33), SUM("ResolutionWidth" + 34), SUM("ResolutionWidth" + 35), SUM("ResolutionWidth" + 36), SUM("ResolutionWidth" + 37), SUM("ResolutionWidth" + 38), SUM("ResolutionWidth" + 39), SUM("ResolutionWidth" + 40), SUM("ResolutionWidth" + 41), SUM("ResolutionWidth" + 42), SUM("ResolutionWidth" + 43), SUM("ResolutionWidth" + 44), SUM("ResolutionWidth" + 45), SUM("ResolutionWidth" + 46), SUM("ResolutionWidth" + 47), SUM("ResolutionWidth" + 48), SUM("ResolutionWidth" + 49), SUM("ResolutionWidth" + 50), SUM("ResolutionWidth" + 51), SUM("ResolutionWidth" + 52), SUM("ResolutionWidth" + 53), SUM("ResolutionWidth" + 54), SUM("ResolutionWidth" + 55), SUM("ResolutionWidth" + 56), SUM("ResolutionWidth" + 57), SUM("ResolutionWidth" + 58), SUM("ResolutionWidth" + 59), SUM("ResolutionWidth" + 60), SUM("ResolutionWidth" + 61), SUM("ResolutionWidth" + 62), SUM("ResolutionWidth" + 63), SUM("ResolutionWidth" + 64), SUM("ResolutionWidth" + 65), SUM("ResolutionWidth" + 66), SUM("ResolutionWidth" + 67), SUM("ResolutionWidth" + 68), SUM("ResolutionWidth" + 69), SUM("ResolutionWidth" + 70), SUM("ResolutionWidth" + 71), SUM("ResolutionWidth" + 72), SUM("ResolutionWidth" + 73), SUM("ResolutionWidth" + 74), SUM("ResolutionWidth" + 75), SUM("ResolutionWidth" + 76), SUM("ResolutionWidth" + 77), SUM("ResolutionWidth" + 78), SUM("ResolutionWidth" + 79), SUM("ResolutionWidth" + 80), SUM("ResolutionWidth" + 81), SUM("ResolutionWidth" + 82), SUM("ResolutionWidth" + 83), SUM("ResolutionWidth" + 84), SUM("ResolutionWidth" + 85), SUM("ResolutionWidth" + 86), SUM("ResolutionWidth" + 87), SUM("ResolutionWidth" + 88), SUM("ResolutionWidth" + 89) FROM logs; +SELECT "SearchEngineID", "ClientIP", COUNT(*) AS c, SUM("IsRefresh"), AVG("ResolutionWidth") FROM logs WHERE "SearchPhrase" <> '' GROUP BY "SearchEngineID", "ClientIP" ORDER BY c DESC LIMIT 10; +SELECT "WatchID", "ClientIP", COUNT(*) AS c, SUM("IsRefresh"), AVG("ResolutionWidth") FROM logs WHERE "SearchPhrase" <> '' GROUP BY "WatchID", "ClientIP" ORDER BY c DESC LIMIT 10; +SELECT "WatchID", "ClientIP", COUNT(*) AS c, SUM("IsRefresh"), AVG("ResolutionWidth") FROM logs GROUP BY "WatchID", "ClientIP" ORDER BY c DESC LIMIT 10; +SELECT "URL", COUNT(*) AS c FROM logs GROUP BY "URL" ORDER BY c DESC LIMIT 10; +SELECT 1, "URL", COUNT(*) AS c FROM logs GROUP BY 1, "URL" ORDER BY c DESC LIMIT 10; +SELECT "ClientIP", "ClientIP" - 1, "ClientIP" - 2, "ClientIP" - 3, COUNT(*) AS c FROM logs GROUP BY "ClientIP", "ClientIP" - 1, "ClientIP" - 2, "ClientIP" - 3 ORDER BY c DESC LIMIT 10; +SELECT "URL", COUNT(*) AS PageViews FROM logs WHERE "CounterID" = 62 AND "EventDate" >= 15887 AND "EventDate" <= 15917 AND "DontCountHits" = 0 AND "IsRefresh" = 0 AND "URL" <> '' GROUP BY "URL" ORDER BY PageViews DESC LIMIT 10; +SELECT "Title", COUNT(*) AS PageViews FROM logs WHERE "CounterID" = 62 AND "EventDate" >= 15887 AND "EventDate" <= 15917 AND "DontCountHits" = 0 AND "IsRefresh" = 0 AND "Title" <> '' GROUP BY "Title" ORDER BY PageViews DESC LIMIT 10; +SELECT "URL", COUNT(*) AS PageViews FROM logs WHERE "CounterID" = 62 AND "EventDate" >= 15887 AND "EventDate" <= 15917 AND "IsRefresh" = 0 AND "IsLink" <> 0 AND "IsDownload" = 0 GROUP BY "URL" ORDER BY PageViews DESC LIMIT 10 OFFSET 1000; +SELECT "TraficSourceID", "SearchEngineID", "AdvEngineID", CASE WHEN ("SearchEngineID" = 0 AND "AdvEngineID" = 0) THEN "Referer" ELSE '' END AS Src, "URL" AS Dst, COUNT(*) AS PageViews FROM logs WHERE "CounterID" = 62 AND "EventDate" >= 15887 AND "EventDate" <= 15917 AND "IsRefresh" = 0 GROUP BY "TraficSourceID", "SearchEngineID", "AdvEngineID", Src, Dst ORDER BY PageViews DESC LIMIT 10 OFFSET 1000; +SELECT "URLHash", "EventDate", COUNT(*) AS PageViews FROM logs WHERE "CounterID" = 62 AND "EventDate" >= 15887 AND "EventDate" <= 15917 AND "IsRefresh" = 0 AND "TraficSourceID" IN (-1, 6) AND "RefererHash" = 3594120000172545465 GROUP BY "URLHash", "EventDate" ORDER BY PageViews DESC LIMIT 10 OFFSET 100; +SELECT "WindowClientWidth", "WindowClientHeight", COUNT(*) AS PageViews FROM logs WHERE "CounterID" = 62 AND "EventDate" >= 15887 AND "EventDate" <= 15917 AND "IsRefresh" = 0 AND "DontCountHits" = 0 AND "URLHash" = 2868770270353813622 GROUP BY "WindowClientWidth", "WindowClientHeight" ORDER BY PageViews DESC LIMIT 10 OFFSET 10000; +SELECT DATE_TRUNC('minute', ts) AS M, COUNT(*) AS PageViews FROM logs WHERE "CounterID" = 62 AND "EventDate" >= 15900 AND "EventDate" <= 15901 AND "IsRefresh" = 0 AND "DontCountHits" = 0 GROUP BY DATE_TRUNC('minute', ts) ORDER BY DATE_TRUNC('minute', ts) LIMIT 10 OFFSET 1000; diff --git a/ravel/query b/ravel/query new file mode 100755 index 0000000000..94dabcfb29 --- /dev/null +++ b/ravel/query @@ -0,0 +1,32 @@ +#!/bin/bash +# Reads one SQL statement from stdin and runs it against ravel-server. +# Stdout: query result. +# Stderr: query runtime in fractional seconds on the last line. +# Exit non-zero on error, which the driver records as `null`. +set -e +. ./ravel-env.sh + +query=$(cat) + +# The SQL endpoint's `start`/`end` default to the last hour. ClickBench's hits +# rows carry 2013 EventTime values, so every query has to state a window that +# spans the whole dataset or it would legitimately match nothing. +body=$(jq -n --arg q "$query" '{query: $q, start: 0, end: 4102444800, timeout: 3600}') + +t1=$(date +%s%3N) +out=$(curl -sS --fail-with-body -X POST \ + -H 'Content-Type: application/json' \ + -H "x-ravel-tenant: $RAVEL_TENANT" \ + --data-binary "$body" \ + "http://${RAVEL_HTTP}/api/v1/sql") && exit_code=0 || exit_code=$? +t2=$(date +%s%3N) + +if [ "$exit_code" -ne 0 ]; then + printf '%s\n' "$out" >&2 + exit "$exit_code" +fi + +printf '%s\n' "$out" + +duration=$((t2 - t1)) +awk -v d="$duration" 'BEGIN { printf "%.3f\n", d / 1000 }' >&2 diff --git a/ravel/ravel-env.sh b/ravel/ravel-env.sh new file mode 100644 index 0000000000..3388d72ebe --- /dev/null +++ b/ravel/ravel-env.sh @@ -0,0 +1,63 @@ +# Shared environment for the Ravel ClickBench entry. Sourced by every script. +# +# Ravel keeps every durable byte in S3-compatible object storage; there is no +# local-disk storage mode (see README.md). The bucket is supplied by the +# operator and the credentials come from the EC2 instance profile, so no key +# ever appears in this repository or in a process argument list. +# shellcheck shell=bash + +: "${RAVEL_S3_BUCKET:?set RAVEL_S3_BUCKET to a bucket the instance role may read and write}" +export RAVEL_S3_BUCKET +export RAVEL_S3_REGION="${RAVEL_S3_REGION:-us-east-1}" + +# ADR-0106: fetch short-lived credentials from IMDSv2. The server refuses to +# start if an inline credential flag is set alongside this, so scrub every +# credential variable an interactive shell may have exported; a stray one would +# either fail startup or silently make the run use static keys instead of the +# instance role, which is the thing this entry is documenting. +export RAVEL_S3_AUTH=instance-role +unset RAVEL_S3_ACCESS_KEY RAVEL_S3_SECRET_KEY RAVEL_S3_SESSION_TOKEN \ + RAVEL_S3_ACCESS_KEY_ID RAVEL_S3_SECRET_ACCESS_KEY \ + AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN + +# The AWS CLI (used only by ./data-size) reads the same instance profile. +export AWS_DEFAULT_REGION="$RAVEL_S3_REGION" + +export RAVEL_TENANT="${RAVEL_TENANT:-clickbench}" +# Must match what ./load provisioned: a server configured for a different shard +# count refuses to resolve rather than answering over a subset (ADR-0050 s5). +export RAVEL_SHARDS="${RAVEL_SHARDS:-4}" + +# Where ./install puts the binaries. +RAVEL_BIN_DIR="${RAVEL_BIN_DIR:-$PWD/bin}" +export RAVEL_BIN_DIR +export RAVEL_SERVER="$RAVEL_BIN_DIR/ravel-server" +export RAVEL_CLI="$RAVEL_BIN_DIR/ravel-cli" + +# ADR-0046 read cache, local-disk tier. Empty (the default) means the RAM tier +# only, which is what the published numbers use. +# +# Off by default because on the ClickBench reference machine it is a +# pessimisation, measured rather than assumed. A c6a.4xlarge has no instance +# store, so "local disk" is a 500 GB gp2 volume: 286 MB/s sequential here, and +# gp2 caps at 250 MB/s sustained. The same server reads S3 at 855 MB/s +# sustained, 1,117 MB/s peak. Caching an object to this volume therefore makes +# the next read of it about 3x slower than fetching it again from S3. +# +# Set it to a path when the disk is genuinely faster than the store: an +# instance-store box (i4i, c6ad, c7gd: NVMe at multiple GB/s) or a gp3 volume +# with provisioned throughput. The disk tier is bounded by the same resolved +# ceiling as the RAM tier, 26.3 GB on this host, so the whole 11.24 GB corpus +# fits either way. +export RAVEL_CACHE_DIR="${RAVEL_CACHE_DIR:-}" + +# Loopback only: --dev-insecure-tenant-header refuses to enable unless +# --listen-http binds a loopback address, so this stays a single-node local +# benchmark and does not weaken the server's auth posture on any reachable +# interface. +export RAVEL_HTTP="${RAVEL_HTTP:-127.0.0.1:9080}" + +# ./start records the server's pid here and ./stop reads it. Stopping by pid +# rather than by command-line pattern keeps ./stop from matching, and killing, +# an unrelated process whose argv merely contains the server's flags. +export RAVEL_PIDFILE="${RAVEL_PIDFILE:-$PWD/server.pid}" diff --git a/ravel/results/20260907/c6a.4xlarge.json b/ravel/results/20260907/c6a.4xlarge.json new file mode 100644 index 0000000000..46ec006e43 --- /dev/null +++ b/ravel/results/20260907/c6a.4xlarge.json @@ -0,0 +1,234 @@ +{ + "system": "Ravel", + "date": "2026-09-07", + "machine": "c6a.4xlarge", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": [ + "Rust", + "column-oriented", + "stateless", + "S3" + ], + "load_time": 1220.49, + "data_size": 12247787064, + "result": [ + [ + 0.196, + 0.13, + 0.139 + ], + [ + 14.702, + 12.142, + 11.417 + ], + [ + 11.921, + 12.245, + 11.244 + ], + [ + 11.303, + 11.603, + 11.932 + ], + [ + 11.879, + 11.516, + 11.745 + ], + [ + 12.23, + 11.729, + 11.103 + ], + [ + 2.302, + 2.02, + 2.004 + ], + [ + 10.902, + 10.898, + 10.835 + ], + [ + 14.218, + 11.498, + 11.24 + ], + [ + 11.438, + 11.246, + 11.316 + ], + [ + 11.0, + 10.788, + 10.888 + ], + [ + 11.202, + 11.044, + 11.311 + ], + [ + 10.984, + 11.281, + 11.411 + ], + [ + 12.044, + 11.063, + 11.374 + ], + [ + 10.84, + 11.438, + 11.146 + ], + [ + 11.117, + 11.059, + 11.312 + ], + [ + 11.74, + 12.154, + 13.353 + ], + [ + 11.529, + 11.258, + 11.726 + ], + [ + 12.17, + 11.646, + 11.742 + ], + [ + 21.863, + 14.055, + 9.458 + ], + [ + 11.489, + 11.686, + 11.766 + ], + [ + 11.905, + 11.995, + 12.147 + ], + [ + 14.76, + 14.631, + 15.426 + ], + [ + 10.05, + 9.707, + 9.867 + ], + [ + 9.422, + 9.132, + 9.123 + ], + [ + 9.171, + 9.216, + 9.083 + ], + [ + 10.343, + 9.213, + 9.005 + ], + [ + 13.529, + 13.468, + 12.944 + ], + [ + 15.462, + 14.662, + 15.751 + ], + [ + 12.433, + 12.33, + 12.398 + ], + [ + 12.285, + 12.548, + 12.495 + ], + [ + 12.337, + 12.458, + 12.499 + ], + [ + null, + null, + null + ], + [ + 14.26, + 13.813, + 13.056 + ], + [ + 13.024, + 13.063, + 13.121 + ], + [ + 11.665, + 12.233, + 12.052 + ], + [ + 11.303, + 9.122, + 8.675 + ], + [ + 11.41, + 8.843, + 8.647 + ], + [ + 11.332, + 8.986, + 8.952 + ], + [ + 11.506, + 8.932, + 8.624 + ], + [ + 11.456, + 8.964, + 9.049 + ], + [ + 11.141, + 9.203, + 9.048 + ], + [ + 11.445, + 8.988, + 8.646 + ] + ] +} diff --git a/ravel/results/20260908/c6a.4xlarge.tuned.json b/ravel/results/20260908/c6a.4xlarge.tuned.json new file mode 100644 index 0000000000..02616679c5 --- /dev/null +++ b/ravel/results/20260908/c6a.4xlarge.tuned.json @@ -0,0 +1,234 @@ +{ + "system": "Ravel", + "date": "2026-09-08", + "machine": "c6a.4xlarge", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "yes", + "tags": [ + "Rust", + "column-oriented", + "stateless", + "S3" + ], + "load_time": 1220.49, + "data_size": 12247787064, + "result": [ + [ + 0.213, + 0.13, + 0.143 + ], + [ + 6.433, + 2.779, + 2.601 + ], + [ + 4.662, + 2.878, + 2.872 + ], + [ + 4.321, + 2.726, + 2.761 + ], + [ + 4.722, + 3.061, + 3.313 + ], + [ + 5.243, + 3.5, + 3.614 + ], + [ + 2.102, + 2.237, + 2.208 + ], + [ + 8.741, + 10.387, + 7.456 + ], + [ + 6.483, + 3.65, + 3.853 + ], + [ + 6.859, + 4.225, + 4.285 + ], + [ + 6.338, + 3.407, + 3.8 + ], + [ + 6.125, + 3.446, + 3.362 + ], + [ + 5.013, + 3.338, + 3.381 + ], + [ + 6.917, + 4.141, + 4.098 + ], + [ + 5.219, + 3.551, + 3.531 + ], + [ + 4.957, + 3.508, + 3.642 + ], + [ + 7.521, + 4.591, + 4.586 + ], + [ + 7.201, + 4.548, + 4.447 + ], + [ + 8.054, + 5.704, + 5.458 + ], + [ + 3.279, + 0.782, + 0.744 + ], + [ + 6.281, + 4.086, + 4.038 + ], + [ + 7.211, + 5.154, + 4.644 + ], + [ + 10.075, + 7.493, + 7.546 + ], + [ + 4.613, + 2.588, + 2.635 + ], + [ + 2.979, + 1.327, + 1.202 + ], + [ + 3.292, + 1.273, + 1.186 + ], + [ + 2.849, + 1.311, + 1.315 + ], + [ + 7.016, + 5.138, + 5.145 + ], + [ + 9.772, + 7.393, + 7.462 + ], + [ + 4.783, + 2.767, + 2.729 + ], + [ + 6.758, + 4.534, + 4.97 + ], + [ + 8.466, + 4.916, + 4.688 + ], + [ + 11.509, + 8.236, + 8.108 + ], + [ + 8.775, + 6.84, + 6.967 + ], + [ + 9.254, + 6.839, + 6.927 + ], + [ + 4.69, + 3.122, + 3.077 + ], + [ + 1.939, + 0.384, + 0.344 + ], + [ + 1.975, + 0.391, + 0.368 + ], + [ + 2.022, + 0.371, + 0.363 + ], + [ + 2.7, + 0.613, + 0.671 + ], + [ + 2.0, + 0.403, + 0.364 + ], + [ + 2.11, + 0.342, + 0.34 + ], + [ + 1.875, + 0.321, + 0.296 + ] + ] +} diff --git a/ravel/start b/ravel/start new file mode 100755 index 0000000000..69d07140ea --- /dev/null +++ b/ravel/start @@ -0,0 +1,99 @@ +#!/bin/bash +set -eu +. ./ravel-env.sh + +# Idempotent: ./check succeeds only for the server this harness launched (it +# verifies the pidfile's pid owns the listening port), so "already serving" +# here means ours is up, never a stale one. +if ./check >/dev/null 2>&1; then + exit 0 +fi + +# Nothing of ours is serving. Before launching, the listen ports must be free: +# a killed server holds them for a while after its pid is gone, and a server +# this harness did not start may hold them indefinitely. Launching into either +# fails with EADDRINUSE, and the driver would then time out its check loop +# without saying why. Wait for the transient case, refuse the other. +http_port="${RAVEL_HTTP##*:}" +ports_busy() { + ss -ltnH "( sport = :$http_port or sport = :4317 )" 2>/dev/null | grep -q . +} +for _ in $(seq 1 30); do + ports_busy || break + sleep 1 +done +if ports_busy; then + echo "start: :$http_port or :4317 is still held after 30s by a process this harness did not start:" >&2 + ss -ltnpH "( sport = :$http_port or sport = :4317 )" 2>/dev/null | sed 's/^/start: /' >&2 + echo "start: refusing to launch behind it; stop that server first" >&2 + exit 1 +fi + +# No performance flags. Since Ravel 0.13.0 the server resolves its query +# budgets at startup: fetch concurrency from the core count, the read caches and +# the two SQL memory ceilings from usable memory (MemTotal, capped by the cgroup +# limit in a container), with a fixed segment cap and engine deadline. The +# resolved values and their sources are on the "performance default resolved" +# lines in server.log, which is what a published result should record. +# +# --shards is not a budget: it must match what ./load provisioned, or the server +# refuses to resolve rather than answering over a subset of shards. +# --tenant-hash-unkeyed is a scheme choice, not a performance flag. A fresh +# bucket has no sys/tenancy marker and the server bootstraps one on its first +# start; the default is the keyed derivation, which refuses to start without a +# deployment key file. This benchmark keeps no secret, so it pins the bucket to +# the unkeyed (v1) derivation, which is what makes a tenant's prefix derivable +# and ./data-size possible. It is written once and read on every later start. + +# One log per start, named by UTC time, with server.log a symlink to the +# current one. The driver restarts the server before every cold try, so a +# single rotated .prev kept exactly one prior log and a server that died two +# tries ago had lost its evidence. Readers of server.log (the README's +# "performance default resolved" lines) still find the live server there. +log="server.$(date -u +%Y%m%dT%H%M%SZ).log" +ln -sfn "$log" server.log + +# The disk cache tier is opt-in and off by default; see RAVEL_CACHE_DIR in +# ravel-env.sh for the measurement that says why. +cache_args=() +[ -n "${RAVEL_CACHE_DIR:-}" ] && cache_args=(--cache-dir "$RAVEL_CACHE_DIR") + +# The tuned entry passes its server flags through RAVEL_TUNED_ARGS so the +# setting is reproducible from this file rather than from a hand-edited +# start script. Empty for the stock entry. Word-split on purpose. +# shellcheck disable=SC2206 +tuned_args=(${RAVEL_TUNED_ARGS:-}) + +nohup "$RAVEL_SERVER" \ + --store s3 \ + --s3-auth instance-role \ + --tenant-hash-unkeyed \ + --shards "$RAVEL_SHARDS" \ + --listen-http "$RAVEL_HTTP" \ + --dev-insecure-tenant-header \ + ${cache_args[@]+"${cache_args[@]}"} \ + ${tuned_args[@]+"${tuned_args[@]}"} \ + > "$log" 2>&1 & +pid=$! +echo "$pid" > "$RAVEL_PIDFILE" +disown + +# Wait for it to serve, so that this script returning means the next one can use +# the server. ./check now also confirms that the answering process is this pid, +# so readiness here cannot be satisfied by anything else. The listener opens +# after a startup cache warmup, and the driver is not the only caller: ./load +# restarts the server itself. +for _ in $(seq 1 180); do + if ./check >/dev/null 2>&1; then + exit 0 + fi + if ! kill -0 "$pid" 2>/dev/null; then + echo "start: the server exited during startup, last lines of $log:" >&2 + tail -n 20 "$log" >&2 + exit 1 + fi + sleep 1 +done + +echo "start: the server did not begin serving within 180 seconds; see $log" >&2 +exit 1 diff --git a/ravel/stop b/ravel/stop new file mode 100755 index 0000000000..d14ea2fb82 --- /dev/null +++ b/ravel/stop @@ -0,0 +1,83 @@ +#!/bin/bash +# Stops the server started by ./start, by pid, and does not return until its +# listen ports are free. +# +# This deliberately does not pgrep for the server's command line. The driver +# calls ./stop before every cold try, and a pattern like "ravel-server.*--listen" +# also matches any shell whose own argv happens to contain it — including the +# ssh command of whoever is driving the run, which is a remarkably effective way +# to kill your own session instead of the server. +# +# It also deliberately does not source ravel-env.sh. That file requires +# RAVEL_S3_BUCKET and aborts the shell that sources it when the variable is +# unset, which turned ./stop into a silent no-op that returned 1 and left the +# server running. Stopping a process needs its pid and nothing else; the port +# comes from RAVEL_HTTP if set, else the default the env file would give. +# +# Waiting for the ports, not only the pid, is the part that matters for a cold +# try. A killed server holds its listen sockets during exit for a while after +# `kill -0` stops seeing it; a ./start issued in that window fails with +# EADDRINUSE and the driver, which swallows ./start's exit code, then times out +# its check loop without a cause. A trend runner built on the same pattern lost +# 27 consecutive cold starts to exactly this before it waited on the ports. +set -u +PIDFILE="${RAVEL_PIDFILE:-server.pid}" +http_port="${RAVEL_HTTP:-127.0.0.1:9080}" +http_port="${http_port##*:}" + +ports_busy() { + ss -ltnH "( sport = :$http_port or sport = :4317 )" 2>/dev/null | grep -q . +} +wait_ports_free() { + for _ in $(seq 1 30); do + ports_busy || return 0 + sleep 1 + done + echo "stop: :$http_port or :4317 still held 30s after the server was gone:" >&2 + ss -ltnpH "( sport = :$http_port or sport = :4317 )" 2>/dev/null | sed 's/^/stop: /' >&2 + return 1 +} + +if [ ! -f "$PIDFILE" ]; then + # Nothing of ours is recorded. If the ports are nonetheless held, a server + # this harness does not own is serving, and returning 0 would let the + # driver's next ./start find it and the next "cold" try run warm. Say so + # and fail; the driver's cold cycle tolerates the exit, but ./check will + # refuse the foreign server and the run stops there instead of silently. + if ports_busy; then + echo "stop: no pidfile at $PIDFILE but a server is listening; this harness does not own it:" >&2 + ss -ltnpH "( sport = :$http_port or sport = :4317 )" 2>/dev/null | sed 's/^/stop: /' >&2 + exit 1 + fi + exit 0 +fi + +pid=$(cat "$PIDFILE" 2>/dev/null || true) +case "$pid" in + ''|*[!0-9]*) rm -f "$PIDFILE"; wait_ports_free; exit $? ;; +esac + +# Only signal it if it is still the server: a recycled pid belonging to +# something else must not be killed. +if ! grep -qs 'ravel-server' "/proc/$pid/cmdline" 2>/dev/null; then + rm -f "$PIDFILE" + wait_ports_free + exit $? +fi + +kill "$pid" 2>/dev/null || true +for _ in $(seq 1 30); do + if ! kill -0 "$pid" 2>/dev/null; then + rm -f "$PIDFILE" + wait_ports_free + exit $? + fi + sleep 1 +done +kill -9 "$pid" 2>/dev/null || true +for _ in $(seq 1 10); do + kill -0 "$pid" 2>/dev/null || break + sleep 1 +done +rm -f "$PIDFILE" +wait_ports_free diff --git a/ravel/template.json b/ravel/template.json new file mode 100644 index 0000000000..ac8d3f7d09 --- /dev/null +++ b/ravel/template.json @@ -0,0 +1,12 @@ +{ + "system": "Ravel", + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": [ + "Rust", + "column-oriented", + "stateless", + "S3" + ] +}