From 900aad5ecc96ac506d8487324c2fda1c9f315d18 Mon Sep 17 00:00:00 2001 From: netdatabot <43409846+netdatabot@users.noreply.github.com> Date: Mon, 20 Jul 2026 11:16:03 +0000 Subject: [PATCH] Ingest new documentation --- .../Agent Notifications Reference.mdx | 12 +--- .../Connectors/AWS Kinesis.mdx | 2 +- .../Connectors/Google Cloud Pub Sub.mdx | 2 +- docs/Exporting Metrics/Connectors/JSON.mdx | 2 +- docs/Exporting Metrics/Connectors/MongoDB.mdx | 2 +- .../Exporting Metrics/Connectors/OpenTSDB.mdx | 2 +- .../Exporting Metrics/Exporting reference.mdx | 56 +++++++++++++++++- docs/Exporting Metrics/Prometheus.mdx | 12 ++-- .../Organize systems metrics and alerts.mdx | 58 ++++++++++++++----- docs/Netdata Agent/Logging.mdx | 11 ++++ .../Maintenance/Service Control.mdx | 1 + .../FAQ on Metrics Centralization Points.mdx | 14 +++++ 12 files changed, 139 insertions(+), 35 deletions(-) diff --git a/docs/Alerts & Notifications/Notifications/Agent Dispatched Notifications/Agent Notifications Reference.mdx b/docs/Alerts & Notifications/Notifications/Agent Dispatched Notifications/Agent Notifications Reference.mdx index 06f7681bef..c393c64eff 100644 --- a/docs/Alerts & Notifications/Notifications/Agent Dispatched Notifications/Agent Notifications Reference.mdx +++ b/docs/Alerts & Notifications/Notifications/Agent Dispatched Notifications/Agent Notifications Reference.mdx @@ -109,11 +109,7 @@ Use the `edit-config` script to safely edit configuration files. It automaticall 3. Define recipients per **role** (see below). -4. Restart the Agent for changes to take effect: - - ```bash - sudo systemctl restart netdata - ``` +4. Changes take effect immediately. The next alarm will use the updated configuration, so no restart or reload is required. ## Example: Alert with Role-Based Routing @@ -495,7 +491,7 @@ Here are solutions for common alert notification issues: 1. Verify your service key: ```bash - grep -E "PAGERDUTY_SERVICE_KEY" /etc/netdata/health_alarm_notify.conf + grep -E "DEFAULT_RECIPIENT_PD|SEND_PD" /etc/netdata/health_alarm_notify.conf ``` 2. Test the PagerDuty API directly: @@ -508,9 +504,7 @@ Here are solutions for common alert notification issues: If notifications seem delayed: -1. Check the `delay` parameter in your alarm configuration -2. Verify your `health.d/*.conf` files for delay settings -3. Check the `ALARM_NOTIFY_DELAY` setting in health_alarm_notify.conf +- Check the `delay` parameter in your alarm definitions in `health.d/*.conf` — this is the only per-alert delay setting for notifications; there is no global delay setting in `health_alarm_notify.conf`. ## Related Docs diff --git a/docs/Exporting Metrics/Connectors/AWS Kinesis.mdx b/docs/Exporting Metrics/Connectors/AWS Kinesis.mdx index 83edb6637a..eab75ac391 100644 --- a/docs/Exporting Metrics/Connectors/AWS Kinesis.mdx +++ b/docs/Exporting Metrics/Connectors/AWS Kinesis.mdx @@ -67,7 +67,7 @@ The following options can be defined for this exporter. | password | Password for HTTP authentication | my_password | no | | data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | no | | hostname | The hostname to be used for sending data to the external database server. | [global].hostname | no | -| prefix | The prefix to add to all metrics. | Netdata | no | +| prefix | The prefix to add to all metrics. | netdata | no | | [update every](#option-update-every) | Frequency of sending sending data to the external database, in seconds. | 10 | no | | [buffer on failures](#option-buffer-on-failures) | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | no | | timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 2 * update_every * 1000 | no | diff --git a/docs/Exporting Metrics/Connectors/Google Cloud Pub Sub.mdx b/docs/Exporting Metrics/Connectors/Google Cloud Pub Sub.mdx index 1a4ba3e2ed..c24b555a1f 100644 --- a/docs/Exporting Metrics/Connectors/Google Cloud Pub Sub.mdx +++ b/docs/Exporting Metrics/Connectors/Google Cloud Pub Sub.mdx @@ -55,7 +55,7 @@ The following options can be defined for this exporter. | password | Password for HTTP authentication | my_password | no | | data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | no | | hostname | The hostname to be used for sending data to the external database server. | [global].hostname | no | -| prefix | The prefix to add to all metrics. | Netdata | no | +| prefix | The prefix to add to all metrics. | netdata | no | | [update every](#option-update-every) | Frequency of sending sending data to the external database, in seconds. | 10 | no | | [buffer on failures](#option-buffer-on-failures) | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | no | | timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 2 * update_every * 1000 | no | diff --git a/docs/Exporting Metrics/Connectors/JSON.mdx b/docs/Exporting Metrics/Connectors/JSON.mdx index c5e4a1bcaf..a0210da2e8 100644 --- a/docs/Exporting Metrics/Connectors/JSON.mdx +++ b/docs/Exporting Metrics/Connectors/JSON.mdx @@ -53,7 +53,7 @@ The following options can be defined for this exporter. | password | Password for HTTP authentication | my_password | no | | data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | no | | hostname | The hostname to be used for sending data to the external database server. | [global].hostname | no | -| prefix | The prefix to add to all metrics. | Netdata | no | +| prefix | The prefix to add to all metrics. | netdata | no | | [update every](#option-update-every) | Frequency of sending sending data to the external database, in seconds. | 10 | no | | [buffer on failures](#option-buffer-on-failures) | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | no | | timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 2 * update_every * 1000 | no | diff --git a/docs/Exporting Metrics/Connectors/MongoDB.mdx b/docs/Exporting Metrics/Connectors/MongoDB.mdx index a45c2098ec..953051f426 100644 --- a/docs/Exporting Metrics/Connectors/MongoDB.mdx +++ b/docs/Exporting Metrics/Connectors/MongoDB.mdx @@ -55,7 +55,7 @@ The following options can be defined for this exporter. | password | Password for HTTP authentication | my_password | no | | data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | no | | hostname | The hostname to be used for sending data to the external database server. | [global].hostname | no | -| prefix | The prefix to add to all metrics. | Netdata | no | +| prefix | The prefix to add to all metrics. | netdata | no | | [update every](#option-update-every) | Frequency of sending sending data to the external database, in seconds. | 10 | no | | [buffer on failures](#option-buffer-on-failures) | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | no | | timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 2 * update_every * 1000 | no | diff --git a/docs/Exporting Metrics/Connectors/OpenTSDB.mdx b/docs/Exporting Metrics/Connectors/OpenTSDB.mdx index 479433ac0f..75a1c8639e 100644 --- a/docs/Exporting Metrics/Connectors/OpenTSDB.mdx +++ b/docs/Exporting Metrics/Connectors/OpenTSDB.mdx @@ -53,7 +53,7 @@ The following options can be defined for this exporter. | password | Password for HTTP authentication | my_password | no | | data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | no | | hostname | The hostname to be used for sending data to the external database server. | [global].hostname | no | -| prefix | The prefix to add to all metrics. | Netdata | no | +| prefix | The prefix to add to all metrics. | netdata | no | | [update every](#option-update-every) | Frequency of sending sending data to the external database, in seconds. | 10 | no | | [buffer on failures](#option-buffer-on-failures) | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | no | | timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 2 * update_every * 1000 | no | diff --git a/docs/Exporting Metrics/Exporting reference.mdx b/docs/Exporting Metrics/Exporting reference.mdx index e95fd1601f..b70366811b 100644 --- a/docs/Exporting Metrics/Exporting reference.mdx +++ b/docs/Exporting Metrics/Exporting reference.mdx @@ -63,6 +63,60 @@ Netdata provides three data export modes: | [Prometheus Remote Write](/docs/exporting-metrics/connectors/prometheus-remote-write) | Snappy-compressed protobuf | Binary over HTTP | | [TimescaleDB](https://github.com/netdata/netdata/blob/master/src/exporting/TIMESCALE.md) | JSON streams | Time-series tables | +## How Metric Names Are Constructed + +OpenTSDB and Graphite export dotted metric names. Prometheus scrape and remote write export context-based metric names with labels. + +### Flat-name connectors: OpenTSDB and Graphite + +These connectors join their name components with dots, so the chart and dimension are visible directly in the metric name: + +```text +OpenTSDB: prefix.chart.dimension +Graphite: prefix.hostname.chart.dimension +``` + +- **OpenTSDB** sends the host as a separate tag (`host=...`). +- **Graphite** embeds the host in the dotted path: `prefix.hostname.chart.dimension`. + +Example (OpenTSDB): `netdata.system.cpu.user` with a `host=myhost` tag. + +OpenTSDB and Graphite preserve dots in chart and dimension names. They replace other non-alphanumeric characters with underscores. + +### Prometheus exports: scrape and remote write + +For homogeneous charts, both methods join the prefix and the chart **context** with underscores, and carry the chart, family, and dimension as labels: + +```text +prefix_context{chart="...", family="...", dimension="..."} +``` + +- The chart **context** — the template shared by all charts of the same kind — is part of the metric name, not the individual chart ID or name. +- `average` appends the chart units and `_average`. The scrape endpoint can omit the units with `hideunits=yes`. +- `sum` appends `_sum` without the chart units. +- In `as-collected` mode, incremental and percentage-over-difference counters append `_total`. For charts produced by the Prometheus collector, Netdata does not append `_total`. +- For heterogeneous `as-collected` charts, the dimension moves into the metric name (`prefix_context_dimension`) and is omitted from the labels. +- Netdata sanitizes the context, units, and any dimension embedded in a Prometheus metric name, so dots and other unsupported characters become underscores. Chart, family, and dimension labels retain their label values. The configured prefix is used as provided. + +Example (remote write): `netdata_system_cpu_percentage_average{chart="system.cpu", dimension="user", family="cpu", instance="myhost"}`. + +For the complete naming rules — contexts, units, suffixes, and how to preview the exact metric names via the `allmetrics` endpoint — see the [Prometheus reference](/docs/exporting-metrics/prometheus). + +### Quick comparison + +| Aspect | OpenTSDB / Graphite | Prometheus scrape / remote write | +|:-------------------|:-------------------------------------|:-----------------------------------------------------------------------------------| +| OpenTSDB base | `prefix.chart.dimension` | `prefix_context` | +| Graphite base | `prefix.hostname.chart.dimension` | `prefix_context` | +| Dimension | In the metric name | Label; in the metric name for heterogeneous `as-collected` charts | +| Data-source suffix | None | `_total`, `_average`, or `_sum` when applicable | +| Units in name | No | `average` only | +| Host | Tag (OpenTSDB) or path (Graphite) | `instance` label for remote write and all-host scrape; otherwise the scrape target | +| Sanitization | Chart/dimension preserve dots | Context/units/embedded dimensions replace dots | +| Default prefix | `netdata` | `netdata` | + +Both approaches respect the `send names instead of ids` setting: when enabled, Netdata uses human-friendly chart and dimension names; when disabled, it uses the raw system IDs. See the [OpenTSDB connector options](/docs/exporting-metrics/connectors/opentsdb) for the prefix and name settings. + ## Configuration Structure Your `exporting.conf` file contains these configuration blocks: @@ -86,7 +140,7 @@ Your `exporting.conf` file contains these configuration blocks: enabled = yes destination = localhost:2003 data source = average - prefix = Netdata + prefix = netdata hostname = my-name update every = 10 buffer on failures = 10 diff --git a/docs/Exporting Metrics/Prometheus.mdx b/docs/Exporting Metrics/Prometheus.mdx index 6c64200c05..d38e095d82 100644 --- a/docs/Exporting Metrics/Prometheus.mdx +++ b/docs/Exporting Metrics/Prometheus.mdx @@ -95,7 +95,7 @@ Netdata tracks each Prometheus server's last access time to calculate averages f Like `average` but sums values instead of averaging them. -**Format:** `CONTEXT_UNITS_sum{chart="CHART",family="FAMILY",dimension="DIMENSION"}` +**Format:** `CONTEXT_sum{chart="CHART",family="FAMILY",dimension="DIMENSION"}` To change the data source, add the `source` parameter to the URL: @@ -265,11 +265,11 @@ Or append `&prefix=netdata` to the URL. ### Metric Units -| Source | Unit Behavior | Control | -|:--------------------|:------------------------------------------|:------------------------------------| -| `average` (default) | Adds units to names (e.g., `_KiB_persec`) | `&hideunits=yes` to hide | -| `as-collected` | No units in names | N/A | -| All sources | v1.12+ standardized units | `&oldunits=yes` for pre-v1.12 names | +| Source | Unit Behavior | Control | +|:--------------------|:------------------------------------------|:--------------------------------------------------------------| +| `average` (default) | Adds units to names (e.g., `_KiB_persec`) | `&hideunits=yes` to hide; `&oldunits=yes` for pre-v1.12 names | +| `as-collected` | No units in names | N/A | +| `sum` | No units in names | N/A | ### Accuracy of Average and Sum Data Sources diff --git a/docs/Netdata Agent/Configuration/Organize systems metrics and alerts.mdx b/docs/Netdata Agent/Configuration/Organize systems metrics and alerts.mdx index 21ebae3de5..0bad9cb2ce 100644 --- a/docs/Netdata Agent/Configuration/Organize systems metrics and alerts.mdx +++ b/docs/Netdata Agent/Configuration/Organize systems metrics and alerts.mdx @@ -89,12 +89,12 @@ Host labels help you: Netdata automatically generates host labels when it starts, capturing: -| Label Category | Information Captured | -|----------------|-----------------------------------------------------| -| System Info | Kernel version, OS name and version | +| Label Category | Information Captured | +|----------------|---------------------------------------------------------------------------------------------| +| System Info | Kernel version, OS name and version | | Hardware | CPU architecture, cores, frequency, RAM, disk space, product ID, product name, product type | -| Environment | Container details, Kubernetes node status | -| Infrastructure | Virtualization layer, Parent-child streaming status | +| Environment | Container details, Kubernetes node status | +| Infrastructure | Virtualization layer, Parent-child streaming status | View your automatic labels at `http://HOST-IP:19999/api/v1/info`: @@ -118,7 +118,10 @@ Add your own labels to categorize systems by any criteria you need. ```bash cd /etc/netdata # Replace with your Netdata config directory sudo ./edit-config netdata.conf - ``` + :::note + + On Windows, follow [Edit Configuration Files](/docs/netdata-agent/configuration#edit-configuration-files) instead. The `[host labels]` example, naming rules, and environment variable expansion in steps 2 and 3 below work identically on Windows. + ::: 2. Add a `[host labels]` section: @@ -139,16 +142,16 @@ Add your own labels to categorize systems by any criteria you need. ```text [host labels] - region = ${REGION} - rack = ${RACK:-unknown} - env = ${DEPLOYMENT_ENV:-production} - location = ${DC}-${RACK:-default} + region = $\{REGION} + rack = $\{RACK:-unknown} + env = $\{DEPLOYMENT_ENV:-production} + location = $\{DC}-$\{RACK:-default} ``` - | Syntax | Behavior | - |--------|----------| - | `${VAR}` | Replaced with the value of `VAR`. If unset or empty, the label value becomes `[none]` | - | `${VAR:-default}` | Replaced with the value of `VAR`. If unset or empty, uses `default` | + | Syntax | Behavior | + |-------------------|---------------------------------------------------------------------------------------| + | `${VAR}` | Replaced with the value of `VAR`. If unset or empty, the label value becomes `[none]` | + | `${VAR:-default}` | Replaced with the value of `VAR`. If unset or empty, uses `default` | Environment variables are resolved when labels are loaded or reloaded. You can mix them with literal text (e.g., `${DC}-${RACK}`). @@ -158,10 +161,37 @@ Add your own labels to categorize systems by any criteria you need. netdatacli reload-labels ``` + On Windows, run this through `netdatacli.exe` — see [Using netdatacli](/docs/netdata-agent/maintenance/service-control#windows). + 5. Verify your labels at `http://HOST-IP:19999/api/v1/info` Use custom host labels such as `environment` with [Node Rule-Based Room Assignment](/docs/netdata-cloud/spaces-and-rooms/node-rule-based-room-assignment) to route Kubernetes Nodes into separate Rooms. +### Remove custom labels + +Remove a custom label you no longer need. + +1. Edit your Netdata configuration: + + ```bash + cd /etc/netdata # Replace with your Netdata config directory + sudo ./edit-config netdata.conf + ``` + + On Windows, follow [Edit Configuration Files](/docs/netdata-agent/configuration#edit-configuration-files) instead. + +2. In the `[host labels]` section, delete the line for the label you want to remove. +3. Save the file. +4. Apply the change without restarting Netdata: + + ```bash + netdatacli reload-labels + ``` + + On Windows, run this through `netdatacli.exe` — see [Using netdatacli](/docs/netdata-agent/maintenance/service-control#windows). + +5. Confirm the label is gone at `http://HOST-IP:19999/api/v1/info` + ### Stream labels from Child to Parent In Parent-Child setups, host labels automatically stream from children to the parent node. Access any child's labels through the parent at: diff --git a/docs/Netdata Agent/Logging.mdx b/docs/Netdata Agent/Logging.mdx index 3202a6233c..f970ea94ad 100644 --- a/docs/Netdata Agent/Logging.mdx +++ b/docs/Netdata Agent/Logging.mdx @@ -315,6 +315,17 @@ journalctl MESSAGE_ID=9ce0cb58-ab8b-44df-82c4-bf1ad9ee22de --since "1 hour ago" The Netdata service's processes execute within the `netdata` journal namespace. Common queries: +:::note +If `journalctl` reports insufficient permissions for the `netdata` namespace, run the query with `sudo`. +For persistent access without `sudo`, add the user to the `systemd-journal` group, then log out and back in. +Membership grants read access to all local journal files, not only Netdata logs. + +```bash +sudo usermod -aG systemd-journal "$USER" +``` + +::: + ```bash # Real-time log monitoring journalctl -u netdata --namespace=netdata -f diff --git a/docs/Netdata Agent/Maintenance/Service Control.mdx b/docs/Netdata Agent/Maintenance/Service Control.mdx index 5b40b07b2d..3c64ae3fae 100644 --- a/docs/Netdata Agent/Maintenance/Service Control.mdx +++ b/docs/Netdata Agent/Maintenance/Service Control.mdx @@ -160,5 +160,6 @@ The `netdatacli` tool is available on Windows at the Netdata installation path ( | **Stop** | `Stop-Service Netdata` | Task Manager > Services > Netdata | — | | **Restart** | `Restart-Service Netdata` | Task Manager > Services > Netdata | — | | **Reload Health** | — | — | `netdatacli reload-health` | +| **Reload Labels** | — | — | `netdatacli reload-labels` | | **Shutdown** | — | — | `netdatacli shutdown-agent` | | **Ping** | — | — | `netdatacli ping` | diff --git a/docs/Netdata Parents/Metrics Centralization Points/FAQ on Metrics Centralization Points.mdx b/docs/Netdata Parents/Metrics Centralization Points/FAQ on Metrics Centralization Points.mdx index 53d405a404..a09259c2bb 100644 --- a/docs/Netdata Parents/Metrics Centralization Points/FAQ on Metrics Centralization Points.mdx +++ b/docs/Netdata Parents/Metrics Centralization Points/FAQ on Metrics Centralization Points.mdx @@ -83,6 +83,20 @@ We recommend using Netdata Cloud to avoid receiving duplicate alert notification
+
+Should I configure local collectors (like httpcheck) on both Parents in an HA cluster?
+ +Local collectors such as `httpcheck` are synthetic checks that run on the Netdata node where they are configured. They do not run against data streamed from child nodes. Each Parent in a cluster is a full Netdata node, so a local collector only runs on the Parent where you configure it. + +If you configure a collector on only one Parent and that Parent goes down, the check stops entirely until that Parent recovers, even though your child metrics continue flowing through the other Parent. + +To make synthetic checks resilient to a Parent failure, configure the collector on both Parents. In an active-active cluster, each Parent also replicates its local collector metrics to the other, so both Parents maintain a complete copy. + +Be aware that each Parent evaluates alerts for its own local collectors independently. Since each Parent is a separate Netdata node, the same check configured on both produces alerts from two different nodes. This differs from child-node alerts, which Netdata Cloud can deduplicate when multiple Parents evaluate the same child (see the question above on duplicate alert notifications). Consider this trade-off when deciding whether to run the same synthetic check on both Parents. + +
+
+
When I have only Parents connected to Netdata Cloud, will I be able to use the Functions feature on my child nodes?