diff --git a/docs/specs/prw/remote_write_spec_2_0.md b/docs/specs/prw/remote_write_spec_2_0.md index 454aaf6e8..1b81cfea9 100644 --- a/docs/specs/prw/remote_write_spec_2_0.md +++ b/docs/specs/prw/remote_write_spec_2_0.md @@ -4,9 +4,9 @@ nav_title: "2.0" sort_rank: 2 --- -* Version: 2.0-rc.5 +* Version: 2.0-rc.6 * Status: **Experimental** -* Date: May 2024 +* Date: October 2026 The Remote-Write specification, in general, is intended to document the standard for how Prometheus and Prometheus Remote-Write compatible senders send data to Prometheus or Prometheus Remote-Write compatible receivers. @@ -44,6 +44,7 @@ In this document, the following definitions are followed: * a `Wire Format` is the format of the data as it travels on the wire (i.e. in a network). In the case of Remote-Write, this is always the compressed binary protobuf format. * a `Sender` is something that sends Remote-Write data. * a `Receiver` is something that receives (writes) Remote-Write data. The meaning of `Written` is up to the Receiver e.g. usually it means storing received data in a database, but also just validating, splitting or enhancing it. +* a `Mode` is a set of additional rules on top of `Protocol` that tightens optional sender and receiver semantics into a deterministic contract. * `Written` refers to data the `Receiver` has received and is accepting. Whether or not it has ingested this data to persistent storage, written it to a WAL, etc. is up to the `Receiver`. The only distinction is that the `Receiver` has accepted this data rather than explicitly rejecting it with an error response. * a `Sample` is a triplet of (start timestamp, timestamp, value). * a `Histogram` is a triplet of (start timestamp, timestamp, [histogram value](https://github.com/prometheus/docs/blob/b9657b5f5b264b81add39f6db2f1df36faf03efe/content/docs/concepts/native_histograms.md)). @@ -96,9 +97,10 @@ Content encoding request header MUST follow [the RFC 9110](https://www.rfc-edito ``` Content-Type: application/x-protobuf Content-Type: application/x-protobuf;proto= +Content-Type: application/x-protobuf;proto=;mode= ``` -Content type request header MUST follow [the RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-content-type). Senders MUST use `application/x-protobuf` as the only media type. Senders MAY add `;proto=` parameter to the header's value to indicate the fully qualified name of the Protobuf Message that was used, from the two mentioned above. As a result, Senders MUST send any of the three supported header values: +Content type request header MUST follow [the RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-content-type). Senders MUST use `application/x-protobuf` as the only media type. Senders MAY add `;proto=` parameter to the header's value to indicate the fully qualified name of the Protobuf Message that was used, from the two mentioned above. Senders MAY also add `;mode=` parameter to indicate the mode for the Protobuf Message, if any is defined for it. As a result, Senders MUST send one of the following supported header values: For the deprecated message introduced in PRW 1.0, identified by `prometheus.WriteRequest`: @@ -108,10 +110,12 @@ For the deprecated message introduced in PRW 1.0, identified by `prometheus.Writ For the message introduced in PRW 2.0, identified by `io.prometheus.write.v2.Request`: * `Content-Type: application/x-protobuf;proto=io.prometheus.write.v2.Request` +* `Content-Type: application/x-protobuf;proto=io.prometheus.write.v2.Request;mode=stateless` (see [Modes](#modes)) +* `Content-Type: application/x-protobuf;proto=io.prometheus.write.v2.Request;mode=stateless-typed` (see [Modes](#modes)) -When talking to 1.x Receivers, Senders SHOULD use `Content-Type: application/x-protobuf` for backward compatibility. Otherwise, Senders SHOULD use `Content-Type: application/x-protobuf;proto=io.prometheus.write.v2.Request`. More Protobuf Messages might come in 2.x or beyond. +When talking to 1.x Receivers, Senders SHOULD use `Content-Type: application/x-protobuf` for backward compatibility. Otherwise, Senders SHOULD use `Content-Type: application/x-protobuf;proto=io.prometheus.write.v2.Request` (optionally with `;mode=stateless` or `;mode=stateless-typed`). More Protobuf Messages or modes might come in 2.x or beyond. -Receivers MUST use the content type header to identify the Protobuf Message schema to use. Accidental wrong schema choices may result in non-deterministic behaviour (e.g. corruptions). +Receivers MUST use the content type header to identify the Protobuf Message schema and mode to use. Accidental wrong schema choices may result in non-deterministic behaviour (e.g. corruptions). > NOTE: Thanks to reserved fields in [`io.prometheus.write.v2.Request`](#protobuf-message), Receiver accidental use of wrong schema with `prometheus.WriteRequest` will result in empty message. This is generally for convenience to avoid surprising errors, but don't rely on it -- future Protobuf Messages might not have this feature. @@ -190,7 +194,7 @@ Senders SHOULD expect [400 HTTP Bad Request](https://www.rfc-editor.org/rfc/rfc9 #### Invalid Samples -Receivers MAY NOT support certain metric types or samples (e.g. a Receiver might reject sample without metadata type specified or without start timestamp, while another Receiver might accept such sample.). It’s up to the Receiver what sample is invalid. Receivers MUST return a [400 HTTP Bad Request](https://www.rfc-editor.org/rfc/rfc9110.html#name-400-bad-request) status code for write requests that contain any invalid samples unless the [partial retriable write](#retries-on-partial-writes) occurs. +Receivers MAY reject samples that violate this specification, the negotiated [mode](#modes), or receiver-specific limits (such as label count or length limits). Receivers MUST return a [400 HTTP Bad Request](https://www.rfc-editor.org/rfc/rfc9110.html#name-400-bad-request) status code for write requests that contain any invalid samples unless the [partial retriable write](#retries-on-partial-writes) occurs. Senders MUST NOT retry on a 4xx HTTP status codes (other than [429](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429)), which MUST be used by Receivers to indicate that the write operation will never be able to succeed and should not be retried. Senders MAY retry on the 415 HTTP status code with a different content type or encoding to see if the Receiver supports it. @@ -229,9 +233,6 @@ The 2.x protocol is breaking compatibility with 1.x by introducing a new, mandat The `io.prometheus.write.v2.Request` references the new Protobuf Message that's meant to replace and deprecate the Remote-Write 1.0's `prometheus.WriteRequest` message. - The full schema and source of the truth is in Prometheus repository in [`prompb/io/prometheus/write/v2/types.proto`](https://github.com/prometheus/prometheus/blob/main/prompb/io/prometheus/write/v2/types.proto). The `gogo` dependency and options CAN be ignored ([will be removed eventually](https://github.com/prometheus/prometheus/issues/11908)). They are not part of the specification as they don't impact the serialized format. The simplified version of the new `io.prometheus.write.v2.Request` is presented below. @@ -378,7 +379,7 @@ Rationales: https://github.com/prometheus/proposals/blob/alexg/remote-write-20-p -* `metadata` sub-fields SHOULD be provided. Receivers MAY reject series with unspecified `Metadata.type`. +* `metadata` sub-fields SHOULD be provided (in [`stateless-typed` mode](#stateless-typed-mode), `Metadata.type` MUST be specified). * Exemplars SHOULD be provided if they exist for a series. The following subsections define some schema elements in detail. @@ -423,7 +424,7 @@ Rationales: https://github.com/prometheus/proposals/blob/alexg/remote-write-20-p --> Senders MUST send `samples` (or `histograms`) for any given `TimeSeries` in timestamp order. Senders MAY send multiple requests for different series in parallel. -Sample's or Histogram's `start_timestamp` SHOULD be provided for types that follow counter semantics (e.g. counters and counter histograms). Receivers MAY reject those series without `start_timestamp` being set. Given optionality, the 0 value MUST be treated by receivers as unset value. To represent the unlikely 0 unix timestamp in milliseconds, "1" or "-1" value MUST be used. +Sample's or Histogram's `start_timestamp` SHOULD be provided for types that follow counter semantics (e.g. counters and counter histograms; in [`stateless-typed` mode](#stateless-typed-mode), `start_timestamp` MUST be set for these types). Given optionality in default and `stateless` modes, the 0 value MUST be treated by receivers as unset value. To represent the unlikely 0 unix timestamp in milliseconds, "1" or "-1" value MUST be used.