Enhance Stackdog with AI tools, deduplication, and IP management - #110
Merged
Conversation
…s and flags deviations using Isolation Forest
Fix install.sh pointing at wrong repo (vsilent → trydirect)
…53k entries per pass. AI requests are bounded — analyzer.rs:94 takes a timeout_secs and builds the client with .timeout(...); 0 opts out for very slow local models. Default 300 s, override with STACKDOG_AI_TIMEOUT_SECS (config.rs, new ai_timeout_secs field wired through create_analyzer)
Wire AlertDeduplicator into Reporter so identical alerts within the 5-minute window are suppressed (both DB persist and notifications). Add STACKDOG_TRUSTED_CONTAINERS env var to skip posture checks for known-good containers like zabbix_agent.
When the source IP is a trusted proxy (default: private ranges), extract the real client IP from X-Forwarded-For / X-Real-IP headers before banning. Also expand should_auto_ban to cover credential scanning, SQL injection probing, SSRF, and sensitive file access detector IDs. New env vars: - STACKDOG_TRUSTED_PROXY_RANGES (default: 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16)
The fingerprint included source_id and sample_line in the message, so the same Redis/.env alert from different containers produced different fingerprints. Now dedup keys on alert_type + severity + description.
AI analyzer produces sample_lines with summaries instead of actual log lines, so extract_ip_candidates found no IPs. Now falls back to scanning the original entries when sample_line has no IPs. Also expand should_auto_ban keywords to match AI-generated descriptions: rejected connection, coordinated attack, probing, scanning, targeting.
New src/tools/ module with AI function-calling support: - types.rs: ToolDef, ToolCall, ToolResult structs - ip_ban.rs: check_ip_status, ban_ip tools - docker.rs: list_containers, get_container_posture tools - alerts.rs: recent_alerts tool - detectors.rs: run_detectors tool - mod.rs: ToolRegistry dispatching to all tools New CLI command: stackdog ban-ip <IP> --duration 30m --reason "..." Parses duration strings (30m, 1h, 2h) and records offense + blocks IP. 19 new tests covering all tools and the registry.
Analyzer now supports OpenAI function calling with up to 5 tool rounds. AI can call check_ip_status, ban_ip, list_containers, get_container_posture, recent_alerts, run_detectors during analysis to gather context. - ChatMessage/ChatChoice updated for tool_calls and finish_reason - LogAnomaly gains suggested_action field (CLI command suggestion) - System prompt instructs AI to suggest ban-ip commands for attacks - Reporter includes suggested_action in alert messages - ToolRegistry wired into SniffOrchestrator with postures - STACKDOG_AI_TOOLS_ENABLED env var (default: true) - All 328 tests pass
- Skip AI when no errors and no detector findings (use pattern analyzer) - Per-source cooldown (STACKDOG_AI_COOLDOWN_SECS, default 300s / 5min) - Single AI call path — tool-use failure goes straight to pattern analyzer instead of trying summarize() again (eliminates double billing) - Run detectors before AI to decide if analysis is needed
The tool-use loop never completed a single round: ToolRegistry::execute
returned results labelled with the tool name instead of the id of the call
they answered, so every request came back 400 ("'tool_call_id' of
'check_ip_status' not found in 'tool_calls' of previous message") and the
analyzer silently fell back to the pattern analyzer. The id is now stamped
in execute(), keeping all six executors unchanged.
Bans were only inspectable over SQL. Adds GET /api/security/bans (optional
status filter, capped limit) and DELETE /api/security/bans/{ip}, backed by
list_offenses() and a new IpBanEngine::unban_ip() that removes the firewall
rule, marks the offense released, and notifies — returning false for an
address with no active block so the endpoint can answer 404.
Also installs sqlite3 in the prod image, since inspecting the database
required a package the image did not ship.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PaYiQzdsosNr2C8L34ZJg1
get_container_info built ContainerInfo::name from Config.Hostname, which is
never what `docker ps` shows: under network_mode host it is the host's
hostname (alerts read "docker:try", the server's name), and otherwise Docker
defaults it to the short container ID (alerts read "container 2ef2d927f45a
[2ef2d927f45a]", the ID twice).
Uses the inspect name instead, trimming Docker's leading slash, and keeps
hostname then short ID as fallbacks so there is always something to print.
This also repairs mail guard targeting: its patterns ("wordpress", "php",
"apache") are matched against ContainerInfo::name, and could not match a
hostname or a short ID.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PaYiQzdsosNr2C8L34ZJg1
Fix/self ingestion dedup toolcalls
insert_offense() writes a new row per detection and mark_blocked() flips every Active row for the address, so one ban leaves several Blocked rows behind. unban_expired() iterated rows, which meant a firewall call and a notification each: an address banned after five offenses produced five identical "Released IP ban" alerts inside one second. Rows are still all marked released, but the firewall call, the alert, and the returned count are now per address. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PaYiQzdsosNr2C8L34ZJg1
There was no way to protect an address from banning. A Hetzner load balancer fronting the service was banned off its own access log: its IP is public, so it does not fall in the RFC1918 defaults of STACKDOG_TRUSTED_PROXY_RANGES, and the X-Forwarded-For lookup that would have found the real client never ran. The balancer's own IP was banned instead, cutting inbound traffic. The allowlist is checked at the top of record_offense, before the offense row is written, so a protected address accumulates no history that a later config change could act on. Accepts bare addresses as well as CIDR, since that is how operators write them. Empty by default. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PaYiQzdsosNr2C8L34ZJg1
insert_offense() wrote a row per detection and left offense_count at 1, so the table grew linearly with every hit from a scanner, and one ban spread across several rows — the reason expiring a ban emitted a notification per row. record_offense_occurrence() upserts on (ip_address, source_type) instead, incrementing offense_count, and record_offense() reads that counter rather than counting rows. The counter restarts when the stored activity predates the find_time window, or when the address has served a ban, so a banned address does not stay one detection away from being banned again. init_database() collapses existing rows before enforcing the layout with a unique index: each group's tally and earliest sighting fold into the row that survives (a live block, else the most recent). The index doubles as the guard that keeps the collapse from running twice, which would recount survivors as groups of one. Verified against a SQLite file with the old layout: five rows for one address became one row with offense_count 5, keeping the Blocked row and the earliest first_seen, unchanged on a second run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PaYiQzdsosNr2C8L34ZJg1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces several significant improvements to Stackdog, focusing on enabling and documenting IP banning support (via iptables/nftables), updating Docker and deployment configurations, and rebranding references from
vsilenttotrydirect. The changes ensure that Stackdog can manage firewall rules from within containers, improve documentation for users and contributors, and update all project links and metadata to reflect the new organization.IP banning support and Docker configuration:
nftablesandiptablesin all backend Dockerfiles (docker/backend/Dockerfile,docker/prod/Dockerfile,docker/local/Dockerfile) to enable IP banning from within containers. Host binaries are not accessible, so these must be present in the image. [1] [2] [3]docker-compose.ymlanddocker-compose.app.ymlto usenetwork_mode: hostandcap_add: NET_ADMINfor thestackdogservice, allowing containers to modify host firewall rules for IP banning. Also added explanatory comments and updated environment variables. [1] [2]docker runcommands and documentation inREADME.mdto use--network hostand--cap-add=NET_ADMIN, and added notes explaining why these flags are necessary for IP banning to work. [1] [2] [3] [4]Configuration and documentation enhancements:
.env.samplefor alert deduplication (STACKDOG_ALERT_DEDUP_WINDOW_SECS), Slack notification customization, and IP ban allowlisting (STACKDOG_IP_BAN_ALLOWLIST). [1] [2]install.sh) to mention prerequisites for IP banning and updated the repository reference totrydirect/stackdog. [1] [2].github/workflows/website-docker.yml) for building and pushing the website Docker image.Rebranding and repo reference updates:
trydirect/stackdoginstead ofvsilent/stackdog, including inREADME.md,CONTRIBUTING.md,BUGS.md,CHANGELOG.md,STATUS.md,docs/INDEX.md,docs/QUICKSTART.md,QWEN.md,DEVELOPMENT.md, and install instructions. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]Alert deduplication improvements:
is_duplicate_keyto theAlertDeduplicatorfor deduplicating alerts based on a custom key, allowing more flexible alert suppression logic.Docker Hub and versioning updates:
vsilent/stackdogtotrydirect/stackdogin CI workflows and documentation, and bumped the crate version to0.2.4inCargo.toml. [1] [2]These changes collectively improve Stackdog's security enforcement capabilities, clarify deployment requirements, and ensure consistency in branding and documentation.