diff --git a/charts/aisix/README.md b/charts/aisix/README.md index 6c0dedb..434eb7a 100644 --- a/charts/aisix/README.md +++ b/charts/aisix/README.md @@ -335,6 +335,10 @@ keda: threshold: "100" ``` +A streamed request is counted in `aisix_llm_requests_total` when the stream +ends, not when it starts, so during a burst of long streams this signal lags by +roughly one stream length. The steady-state rate is unaffected. + `autoscaling` and `keda` are mutually exclusive — enabling both fails the render rather than letting two controllers fight over the replica count. @@ -616,7 +620,7 @@ rejects. New installs set the value in its own key in `values.yaml` or under | keda.minReplicas | int | `2` | Lower replica bound | | keda.pollingInterval | int | `15` | How often KEDA evaluates the triggers, in seconds | | keda.restoreToOriginalReplicaCount | bool | `false` | Restore the original replica count when the ScaledObject is deleted | -| keda.triggers | list | `[]` | KEDA triggers. Required when `keda.enabled` is true. For example: `[{type: prometheus, metadata: {serverAddress: "http://prometheus:9090", query: "sum(rate(aisix_llm_requests_total[2m]))", threshold: "100"}}]` | +| keda.triggers | list | `[]` | KEDA triggers. Required when `keda.enabled` is true. For example: `[{type: prometheus, metadata: {serverAddress: "http://prometheus:9090", query: "sum(rate(aisix_llm_requests_total[2m]))", threshold: "100"}}]`. Streamed requests are counted when the stream ends, so during a burst of long streams this signal lags by roughly one stream length; the steady-state rate is unaffected | | listeners | list | `[]` | Proxy listeners, one entry per port. Empty keeps the single plain-HTTP listener described by `containerPorts.proxy` and `service.port` — see "Serve HTTPS and plain HTTP together" above | | livenessProbe.enabled | bool | `true` | | | livenessProbe.failureThreshold | int | `3` | | diff --git a/charts/aisix/README.md.gotmpl b/charts/aisix/README.md.gotmpl index 26e04b8..e8c20e5 100644 --- a/charts/aisix/README.md.gotmpl +++ b/charts/aisix/README.md.gotmpl @@ -329,6 +329,10 @@ keda: threshold: "100" ``` +A streamed request is counted in `aisix_llm_requests_total` when the stream +ends, not when it starts, so during a burst of long streams this signal lags by +roughly one stream length. The steady-state rate is unaffected. + `autoscaling` and `keda` are mutually exclusive — enabling both fails the render rather than letting two controllers fight over the replica count. diff --git a/charts/aisix/values.yaml b/charts/aisix/values.yaml index 5873ab6..7bd7674 100644 --- a/charts/aisix/values.yaml +++ b/charts/aisix/values.yaml @@ -444,7 +444,10 @@ keda: annotations: {} # -- KEDA triggers. Required when `keda.enabled` is true. For example: # `[{type: prometheus, metadata: {serverAddress: "http://prometheus:9090", - # query: "sum(rate(aisix_llm_requests_total[2m]))", threshold: "100"}}]` + # query: "sum(rate(aisix_llm_requests_total[2m]))", threshold: "100"}}]`. + # Streamed requests are counted when the stream ends, so during a burst of + # long streams this signal lags by roughly one stream length; the + # steady-state rate is unaffected triggers: [] # -- `behavior` for the HPA KEDA creates. Empty uses the Kubernetes defaults behavior: {}