Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/Collecting Metrics/Prometheus Profile Format.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -459,11 +459,11 @@ dimension, presentation, and selector field. Prometheus profiles add these rules
lowers emitted chart cardinality; `aggregation` only selects the value for resulting collisions. Every scraped series is
still processed and retained in the collector's metric store. This chart reduction is separate from Prometheus
relabeling: it does not remove or rewrite stored series labels.
- **Only collected series can be charted.** `*_info` families are skipped. Untyped scalar families are collected only
when the selected profile or job `fallback_type` maps them to a gauge or counter, or when the name ends in `_total`
(the last-resort implicit counter rule). Declared type wins; job policy wins over profile policy; and an explicit
gauge match wins over counter classification within the same policy layer. A chart's `algorithm` acts later and
cannot make an unclassified sample collectible.
- **Only collected series can be charted.** Gauge families named `*_info` are skipped. Untyped scalar families are
collected only when the selected profile or job `fallback_type` maps them to a gauge or counter, or when the name ends
in `_total` (the last-resort implicit counter rule). Declared type wins; job policy wins over profile policy; and an
explicit gauge match wins over counter classification within the same policy layer. A chart's `algorithm` acts later
and cannot make an unclassified sample collectible.
- **Every group that contains charts must list the metrics its selectors reference in its `metrics` list** (or inherit
them from an ancestor group). A selector on a metric outside the group's declared scope fails validation.

Expand Down
38 changes: 32 additions & 6 deletions docs/Developer and Contributor Corner/Netdata Support Bundle.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ slug: "/developer-and-contributor-corner/netdata-support-bundle"

# Netdata Support Bundle

The Netdata Support Bundle is a diagnostic collector that ships with the Netdata Agent. It gathers configuration, logs, process state, and runtime information from a single node into one sanitized archive that you can attach to a support ticket or a GitHub bug report.
The Netdata Support Bundle is a diagnostic collector that ships with the Netdata Agent. It gathers configuration, logs, process state, and runtime information from a single node into one archive for troubleshooting. Standard captures are sanitized; explicitly included SNMP diagnostic files are raw and require private sharing.

It exists so you do not have to run a dozen commands by hand and paste the output into a ticket. One command produces one file that contains what the Netdata team needs to diagnose most problems.

Expand All @@ -25,7 +25,7 @@ The tool ships in two forms with the same behavior and the same archive layout:

- It collects the data support usually asks for, so you avoid several rounds of back and forth.
- It runs at the lowest process priority and stops each command after a timeout, so it does not compete with your production workload.
- It redacts secrets and pseudonymizes IP addresses and hostnames before writing the archive, so the result is safe to share.
- Standard captures redact secrets and pseudonymize IP addresses and hostnames. Review the privacy exceptions below before sharing.
- It still produces useful output when the agent is not running, by reading state and logs from disk.

## Run it on Linux, macOS, BSD, or in a container
Expand Down Expand Up @@ -73,6 +73,7 @@ Both scripts accept the same set of options. The POSIX flags and their PowerShel
| `--since HOURS` | `-SinceHours N` | `24` | How many hours of logs to include. |
| `--timeout SECONDS` | `-TimeoutSeconds N` | `10` | Per-command timeout, so a slow command cannot stall the run. |
| `--no-obfuscate` | `-NoObfuscate` | off | Turn off IP and hostname pseudonymization. Secrets are still redacted. |
| `--include-snmp-diagnostics` | `-IncludeSnmpDiagnostics` | off | Include existing raw SNMP diagnostic files; no sanitization. Share privately. |
| `--keep-staging` | `-KeepStaging` | off | Keep the temporary working directory for inspection. |
| `--selftest` | `-SelfTest` | off | Run the built-in sanitizer tests and exit without collecting anything. |
| `-v`, `--version` | `-Version` | | Print the tool version and exit. |
Expand All @@ -98,15 +99,40 @@ The archive is organized into numbered directories so a person or an automated r
- `08-network/`, local connectivity relevant to the agent.
- `09-permissions/`, file modes, ownership, plugin capabilities, extended attributes, security contexts, and ACLs for the agent's directories and plugins.

## Include SNMP diagnostics

For SNMP metrics, BGP, licensing, or topology issues, run:

```sh
sudo netdata-support-bundle --include-snmp-diagnostics
```

On Windows, add `-IncludeSnmpDiagnostics` to the PowerShell invocation. The same
bundle includes available original SNMP evidence under `06-state/snmp-diagnostics/`.
No extra SNMP requests are sent, and no Python or decompressor is required.

**These files bypass sanitization and pseudonymization.** They may contain
addresses, hostnames, inventory, metric values, and arbitrary device-returned
data. Share this bundle through a restricted support ticket, never a public
GitHub issue. The manifest labels raw files and clears its aggregate redaction
claims when they are included.

Check `06-state/snmp-diagnostics-status.txt` for missing files or copy failures.
Files are copied whole, so bundle size and disk work depend on the retained
evidence. They have independent collection times and may rotate during copying;
a partial bundle can still help support. See
[Collect SNMP troubleshooting data](/docs/network-performance-monitoring/device-metrics/collect-troubleshooting-data)
for retention, publication timing, and missing-file troubleshooting.

## Privacy and sanitization

- Secrets are removed. This covers API tokens, passwords, bearer and basic credentials, private key blocks, and credentials embedded in URLs.
- One deliberate exception: the **streaming API key** is kept as-is in `04-config/stream.conf`, because support needs it to tell whether a child and its parent agree on the same key. Remove or mask that file before sending the bundle if you would rather not share it.
- Standard captures remove secrets. This covers API tokens, passwords, bearer and basic credentials, private key blocks, and credentials embedded in URLs.
- The standard capture exception: the **streaming API key** is kept as-is in `04-config/stream.conf`, because support needs it to tell whether a child and its parent agree on the same key. Remove or mask that file before sending the bundle if you would rather not share it.
- Collected files keep their original bytes, including a byte-order mark and CRLF line endings, so an encoding problem in your configuration is still visible.
- IP addresses and hostnames are replaced with stable pseudonyms by default, so the same address reads as the same pseudonym across the whole bundle. Use `--no-obfuscate` to keep the real values.
- Standard captures replace IP addresses and hostnames with stable pseudonyms by default, so the same address reads as the same pseudonym across those captures. Use `--no-obfuscate` to keep the real values.
- When pseudonymization is on, the map from real values to pseudonyms is written next to the archive, not inside it. Keep that map private and do not attach it to a ticket.
- Review the archive before you send it. The tool runs on your own host and you are in control of what leaves it.

## Attach the bundle

Attach the printed archive file to your support ticket or to a GitHub bug report. The Netdata bug report form points to this same tool, so a bundle is the fastest way to give the team what it needs.
Attach the printed archive file to your support ticket. If it includes raw SNMP diagnostics, use a restricted ticket; otherwise, review it before attaching it to a public GitHub bug report. The Netdata bug report form points to this same tool, so a bundle is the fastest way to give the team what it needs.
2 changes: 1 addition & 1 deletion docs/Netdata AI/Alerts Automation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ learn_status: "Published"
learn_rel_path: "Netdata AI"
keywords: [alerts, automation, ai alerts]
description: "AI-powered alert creation and suggestions"
sidebar_position: "50"
sidebar_position: "60"
learn_link: "https://learn.netdata.cloud/docs/netdata-ai/alerts-automation"
slug: "/netdata-ai/alerts-automation"
---
Expand Down
2 changes: 1 addition & 1 deletion docs/Netdata AI/Anomaly Detection/Anomaly Detection.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/ml-ai/ml-a
sidebar_label: "Anomaly Detection"
learn_status: "Published"
learn_rel_path: "Netdata AI/Anomaly Detection"
sidebar_position: "60"
sidebar_position: "70"
learn_link: "https://learn.netdata.cloud/docs/netdata-ai/anomaly-detection"
slug: "/netdata-ai/anomaly-detection"
---
Expand Down
5 changes: 5 additions & 0 deletions docs/Netdata AI/Conversations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,10 @@ Use the right tool for the right job:

Click the **"Conversations"** button above the space selectors in Netdata Cloud. Your conversation history is saved, allowing you to pick up an investigation where you left off.

## Teaching Netdata AI about your infrastructure

Conversations are also where Netdata AI learns. Tell it what only your team knows — *"remember that this alert is expected on the batch nodes"*, *"db-3 is the replica, not the primary"* — and it keeps that for every future conversation, investigation, and report in your Space. See [Infrastructure Knowledge](/docs/netdata-ai/infrastructure-knowledge).

## AI credits consumption

Usage is based on AI credits (10 monthly complimentary credits on Business plans, plus the ability to top up as needed).
Expand All @@ -82,3 +86,4 @@ Real-Time Conversations are available for all users on a Business plan or free t
- [Investigations](/docs/netdata-ai/investigations) – comprehensive async analysis
- [Insights](/docs/netdata-ai/insights) – on-demand professional reports
- [Troubleshooting](/docs/netdata-ai/troubleshooting) – alert analysis and anomaly exploration
- [Infrastructure Knowledge](/docs/netdata-ai/infrastructure-knowledge) – what Netdata AI knows about your infrastructure, and how to teach it more
75 changes: 75 additions & 0 deletions docs/Netdata AI/Infrastructure Knowledge.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
---
custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/netdata-ai/infrastructure-knowledge.md"
sidebar_label: "Infrastructure Knowledge"
learn_status: "Published"
learn_rel_path: "Netdata AI"
keywords: [infrastructure knowledge, context, memory, infra.md]
description: "Teach Netdata AI about your infrastructure, in a shared document or in conversation"
sidebar_position: "20"
learn_link: "https://learn.netdata.cloud/docs/netdata-ai/infrastructure-knowledge"
slug: "/netdata-ai/infrastructure-knowledge"
---

# Infrastructure Knowledge

Netdata AI sees everything your infrastructure *does* — every metric, every anomaly, every alert. What it cannot see is what your infrastructure *is*: which services matter, which host is the one that is supposed to run hot, who owns what, and what your team considers normal. Without that, "CPU at 91%" is a finding. With it, it may be a machine doing exactly its job.

**Infrastructure Knowledge** is where that context lives. Think of it as onboarding a new SRE to your team: the more you tell Netdata AI about your infrastructure, the more accurate its conversations, investigations, and reports become — and the more its answers sound like they come from someone who already knows your environment.

There are two ways to teach it, and they work together:

- **Write it down.** Your Space has a shared document that describes your infrastructure. Your team edits it directly.
- **Tell it in a conversation.** As you work with Netdata AI, you can ask it to remember things, and it can pick up what you tell it along the way. It gets to know your infrastructure progressively, the way a new colleague would.

Everything you teach Netdata AI is visible under **Manage Space → Infrastructure Knowledge**, in two tabs: **Your Context** and **AI Memory**.

## Your Context

**Your Context** is a single Markdown document your team maintains. It is read at the start of every conversation, investigation, and report about your infrastructure.

Describe the things that only your team knows. The editor offers section templates to get you started:

- **Infrastructure Overview** — what you run and where, in a few paragraphs.
- **Service Tiers** — which services are critical, which are best-effort, what is worth waking someone up for.
- **Known Behaviours** — patterns that look like problems and are not: the nightly batch job, the ML node that always runs hot, the idle half of a blue-green deployment.
- **SLOs & Business Context** — your targets and what a breach means to the business.
- **Team Preferences** — how you want findings presented, who to mention for what.
- **Upcoming Events** — maintenance windows, migrations, expected load changes.
- **Architecture Notes** — naming conventions, dependencies, anything that would trip up a newcomer.

You do not need to fill in everything. A few honest lines about what matters and what is expected already change how Netdata AI reads your data. Skip anything Netdata can already see for itself — node lists, current values, running processes — and never put credentials or personal data in it.

Every save keeps a version, so you can review what changed and go back to an earlier version at any time.

You can also ask Netdata AI to update the document for you in a conversation — for example, *"Add the new Kafka cluster to my infrastructure context"* — and it tells you what it changed.

## AI Memory

**AI Memory** is what Netdata AI has learned from talking to your team. Each memory is a single fact about your infrastructure, shared with everyone in the Space and used in every future conversation, investigation, and report.

You build it up simply by talking:

- *"Remember that the 02:00 disk spike on backup-01 is expected."*
- *"That alert is a false positive — the temp files get cleaned hourly."*
- *"db-3 is the replica, not the primary."*
- *"Keep investigation summaries short."*

Netdata AI tells you when it records something, so you can correct it while you still have the context. Say *"don't remember that"* and it forgets. It may also offer to remember a correction you make during troubleshooting.

The **AI Memory** tab lists everything it has learned. You can read each memory, delete the ones that no longer apply, or clear them all.

## How Netdata AI uses it

Netdata AI reads your context and memories before it interprets anything, not after. Your context tells it what things *mean*; telemetry tells it what is *happening*. It uses your knowledge to interpret the data, never to override it — if your document says a node was decommissioned and the node is still reporting, it tells you the two disagree rather than picking a side.

When your context changes a conclusion — a node it did not flag because you marked it as a staging tier, a spike it called expected because you said so — it says where that came from. When it needed context you have not given it, it says that too, so you know what to add next.

## Availability

Infrastructure Knowledge is available on paid Netdata Cloud plans. Space admins, managers, and troubleshooters can edit it; observers can view it.

## See also

- [Conversations](/docs/netdata-ai/conversations) — where you teach Netdata AI as you work.
- [Investigations](/docs/netdata-ai/investigations) and [AI Insights](/docs/netdata-ai/insights) — reports that use your context.
- [MCP Connections](/docs/netdata-ai/mcp/mcp-connections) — bring in context from the tools your team already runs.
2 changes: 1 addition & 1 deletion docs/Netdata AI/Insights/Insights.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/ml-ai/ai-i
sidebar_label: "Insights"
learn_status: "Published"
learn_rel_path: "Netdata AI/Insights"
sidebar_position: "20"
sidebar_position: "30"
learn_link: "https://learn.netdata.cloud/docs/netdata-ai/insights"
slug: "/netdata-ai/insights"
---
Expand Down
2 changes: 1 addition & 1 deletion docs/Netdata AI/Investigations/Investigations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/netdata-ai
sidebar_label: "Investigations"
learn_status: "Published"
learn_rel_path: "Netdata AI/Investigations"
sidebar_position: "30"
sidebar_position: "40"
learn_link: "https://learn.netdata.cloud/docs/netdata-ai/investigations"
slug: "/netdata-ai/investigations"
---
Expand Down
2 changes: 2 additions & 0 deletions docs/Netdata AI/MCP/MCP Connections/MCP Connections.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ An alert tells you *what* changed. It rarely tells you *why*. That answer usuall

This is the reverse of connecting an AI client *to* Netdata. Here, **Netdata reaches out to your MCP servers**. To instead connect an AI assistant (Claude, Cursor, a CLI) to Netdata's own MCP server, see [Supported AI Clients](/docs/netdata-ai/mcp/supported-ai-clients).

MCP Connections bring in context from other systems. For the context only your team can provide — service tiers, known behaviours, ownership — see [Infrastructure Knowledge](/docs/netdata-ai/infrastructure-knowledge).

![MCP Connections settings](https://raw.githubusercontent.com/netdata/docs-images/refs/heads/master/netdata-cloud/netdata-ai/mcp-connections-settings.png)

## Prerequisites
Expand Down
2 changes: 1 addition & 1 deletion docs/Netdata AI/MCP/MCP.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/netdata-ai
sidebar_label: "MCP"
learn_status: "Published"
learn_rel_path: "Netdata AI/MCP"
sidebar_position: "70"
sidebar_position: "80"
learn_link: "https://learn.netdata.cloud/docs/netdata-ai/mcp"
slug: "/netdata-ai/mcp"
---
Expand Down
4 changes: 4 additions & 0 deletions docs/Netdata AI/Netdata AI.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,10 @@ Local, unsupervised ML runs on every agent, learning normal behavior and scoring

Connect AI clients to Netdata’s MCP server to bring live observability into natural‑language workflows and optional automation. Options include [MCP](/docs/netdata-ai/mcp), [Chat with Netdata](/docs/netdata-ai/mcp/chat-with-netdata), and [Supported AI Clients](/docs/netdata-ai/mcp/supported-ai-clients) like Claude Desktop, Cursor, VS Code, JetBrains IDEs, Claude Code, Gemini CLI, and the Netdata Web Client. Netdata Cloud can also act as an MCP client that connects to your own tools — see [MCP Connections](/docs/netdata-ai/mcp/mcp-connections).

### 8) Infrastructure Knowledge

[Infrastructure Knowledge](/docs/netdata-ai/infrastructure-knowledge) is what Netdata AI knows about *your* infrastructure that telemetry cannot show: service tiers, ownership, known behaviours, SLOs. Write it down in a shared document, or simply tell Netdata AI in a conversation and let it remember — every conversation, investigation, and report uses it.

## Usage and credits

- Eligible Spaces receive 10 free AI credits; each Insights report, investigation, or alert troubleshooting run consumes 1 AI credit.
Expand Down
2 changes: 1 addition & 1 deletion docs/Netdata AI/Troubleshooting/Troubleshooting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/netdata-ai
sidebar_label: "Troubleshooting"
learn_status: "Published"
learn_rel_path: "Netdata AI/Troubleshooting"
sidebar_position: "40"
sidebar_position: "50"
learn_link: "https://learn.netdata.cloud/docs/netdata-ai/troubleshooting"
slug: "/netdata-ai/troubleshooting"
---
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,6 @@ There are no alerts configured by default for this integration.

### Other Problems

#### Collect Live Data for Netdata Support
#### Collect Diagnostics for Netdata Support

For missing SNMP metrics or incomplete SNMP-derived topology, follow [Collect SNMP troubleshooting data](/docs/network-performance-monitoring/device-metrics/collect-troubleshooting-data) to create a raw SNMP data archive that omits credentials and attach it to a restricted Freshdesk ticket.
For SNMP metrics, BGP, licensing, or topology issues, follow [Collect SNMP troubleshooting data](/docs/network-performance-monitoring/device-metrics/collect-troubleshooting-data) to include built-in diagnostics in a support bundle. SNMP evidence is unsanitized; share the bundle through a restricted Netdata Support ticket.
Loading