feat(mcp): JSONL purchase audit log (CUDLY_MCP_AUDIT_LOG)

README.md

@@ -129,16 +129,29 @@ Every other provider's purchase tool (`cudly_aws_savingsplans_purchase`, `cudly_
 - Every real purchase is tagged with a source identifying it came from this MCP server (never a user-suppliable string) and a deterministic idempotency token derived from the request's own parameters. By default, retrying an identical tool call -- however long after the original, and regardless of any clock boundary -- always derives the same token, so the provider dedupes the retry instead of buying twice; this is a fail-safe default, since the worst case of a false dedupe is a skipped intentional repeat, never a double purchase. To deliberately make a second, otherwise-identical purchase (e.g. "buy 3 RIs now" and "buy 3 more next week"), pass a fresh `idempotency_nonce` value on the second call; passing the same nonce on a retry of that same call still dedupes correctly.
 - Provider/SDK failures surface their full error text back to the caller; nothing is swallowed.
 - A **completed** real purchase carries an `archera` block in the response: an optional underutilization-insurance offer (Archera covers the gap if committed capacity goes unused), the signup link, the enrollment window in days, and both partnership disclosures. It is attached only when the purchase actually succeeded, never to a dry run or a failed purchase, because neither started an enrollment window. Archera sponsors CUDly's development from a fraction of their insurance premiums, and CUDly works fully without it; both facts travel with the link in every response so a client rendering this payload cannot present the offer as a neutral recommendation.
-- Every real purchase writes an `mcp purchase ATTEMPT` line and a matching `mcp purchase OK` / `mcp purchase FAILED` line to **stderr**, recording provider, target account, region, resource, count, term, payment option, the resulting commitment ID, and a masked idempotency token. Dry runs are not logged (they spend nothing). Capture your MCP client's stderr if you want this trail retained. Nothing is written to stdout, which the MCP stdio transport owns for JSON-RPC framing.
+- Every real purchase writes an `mcp purchase ATTEMPT` line and a matching `mcp purchase OK` / `mcp purchase FAILED` line to **stderr**, recording provider, target account, region, resource, count, term, payment option, the resulting commitment ID, and a masked idempotency token. Dry runs emit no purchase diagnostic to stderr because they spend nothing, but when auditing is enabled and the append succeeds, they are persisted in the JSONL audit log as `status: "skipped"`. Capture your MCP client's stderr if you want the diagnostic trail retained. Nothing is written to stdout, which the MCP stdio transport owns for JSON-RPC framing.
 
 ### What this server does NOT give you
 
 Understand these before enabling real purchases, especially in a shared or production account:
 
 - **No scheduled/4-eyes approval workflow.** The web UI routes a purchase through `purchase_executions` with a scheduled date and, under 4-eyes mode, a second approver who cannot be the creator. This server has no such workflow: once `CUDLY_MCP_ENABLE_REAL_PURCHASES=1` is set, `dry_run=false` plus `confirm=true` in a single tool call executes immediately. `confirm` is still a guardrail against an accidental call rather than an authorization control -- it is supplied by the model driving the client, not the operator -- but `CUDLY_MCP_ENABLE_REAL_PURCHASES` is the operator-side authorization control layered underneath it: it must be explicitly enabled before *any* tool call, confirmed or not, can spend money on this server.
-- **No persisted audit record.** The CLI writes a `common.AuditRecord` per purchase and the web path persists an execution row; this server writes only the stderr lines above. An MCP purchase does not appear in CUDly's own purchase history, so reconcile against the provider's console/billing data rather than against CUDly.
+- **The persisted audit record is a local JSONL file, not CUDly's purchase history.** When auditing is enabled and a write succeeds, each purchase attempt -- including previews -- is appended as one `common.AuditRecord` JSON line to the path from [Audit log](#audit-log) below. An MCP purchase still does not appear in the CLI/web paths' `purchase_history`, so reconcile against the provider's console/billing data or this file rather than against CUDly's own database.
 - **Credentials are whatever launched the process.** `aws_profile` / `azure_subscription_id` / `gcp_project_id` are per-call arguments chosen by the model, so any account reachable from the ambient credentials is reachable from any tool call. Scope the credentials you launch `cudly-mcp` with to what you are willing to let it spend, rather than relying on the tool arguments to constrain it.
 
+## Audit log
+
+Auditing is **on by default**. When auditing is enabled and a write succeeds, each purchase attempt -- previews included -- appends one `common.AuditRecord` JSON line to a local file, independent of the stderr trail in [Safety model](#safety-model) above.
+
+- **Default path**: `$XDG_STATE_HOME/cudly/mcp-audit.jsonl`, falling back to `~/.local/state/cudly/mcp-audit.jsonl` when `XDG_STATE_HOME` is unset, empty, or relative.
+- **`CUDLY_MCP_AUDIT_LOG`** overrides the path. Setting it to an **empty or whitespace-only string disables the log entirely** -- that is the explicit opt-out; leaving the variable unset is not the same thing and still uses the default path.
+- A preview (`dry_run=true`) is recorded with `status: "skipped"`, `dry_run: true` -- it spent nothing, but it was still a decision worth reconstructing later.
+- When a credential scope is supplied, `credential_scope` records that routing identifier: an AWS profile, Azure subscription, or GCP project. It identifies how CUDly selected the target; it is not a verified provider account ID. Previews may omit it because they do not require a target.
+- A real purchase is recorded `"success"` only when the provider reports `Success: true` with no embedded error. A failed provider call, `Success: false`, or a result containing an error is recorded as `"error"`.
+- Every purchase in one server process shares a single `run_id`, so a log spanning many purchases can be grouped by session.
+- **A write failure never changes the purchase result.** It logs a warning to stderr naming the path and error; the tool response returned to the caller is unaffected.
+- The path is probed for read, append, and directory durability at server startup. MCP creates missing parent directories as owner-only `0700` subject to umask, then opens and syncs every directory edge from the filesystem root. Each ancestor therefore needs search and read permission, every creation parent also needs write permission, and the filesystem must support directory `fsync`. A path that fails these checks stops server construction instead of silently dropping every record for the session.
+
 ## Caveats and known gaps
 
 These are pre-existing behaviours in the underlying purchase clients, not something introduced by or specific to the MCP server -- flagged here so you know what to expect:

cmd/cudly-mcp/main_test.go

@@ -1,20 +1,53 @@
 package main
 
 import (
+	"bytes"
 	"context"
+	"encoding/json"
+	"log"
+	"os"
+	"os/exec"
 	"path/filepath"
 	"strings"
 	"testing"
 	"time"
 
+	"github.com/LeanerCloud/cloud-commitments-go/pkg/common"
 	gosdk "github.com/modelcontextprotocol/go-sdk/mcp"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
+	"golang.org/x/sys/unix"
 
 	cudlymcp "github.com/LeanerCloud/cloud-commitments-mcp"
 	"github.com/LeanerCloud/cloud-commitments-mcp/tools"
 )
 
+const runAsMCPServerEnv = "CUDLY_MCP_TEST_HELPER_PROCESS"
+
+func TestMain(m *testing.M) {
+	if os.Getenv(runAsMCPServerEnv) == "1" {
+		main()
+		os.Exit(0)
+	}
+
+	auditDir, err := os.MkdirTemp("", "cudly-mcp-audit-testmain")
+	if err != nil {
+		log.Printf("create MCP audit test directory: %v", err)
+		os.Exit(1)
+	}
+	if err := os.Setenv(tools.EnvAuditLog, filepath.Join(auditDir, "mcp-audit.jsonl")); err != nil {
+		log.Printf("set MCP audit log test path: %v", err)
+		if cleanupErr := os.RemoveAll(auditDir); cleanupErr != nil {
+			log.Printf("remove MCP audit test directory after setup failure: %v", cleanupErr)
+		}
+		os.Exit(1)
+	}
+
+	code := m.Run()
+	os.RemoveAll(auditDir)
+	os.Exit(code)
+}
+
 // isolateFromAmbientAWS points the AWS SDK at deliberately nonexistent
 // profile/config/credentials so config.LoadDefaultConfig cannot resolve any
 // real credentials -- neither from a dev machine's ~/.aws files nor from the
@@ -38,6 +71,208 @@ func isolateFromAmbientAWS(t *testing.T) {
 	t.Setenv("AWS_WEB_IDENTITY_TOKEN_FILE", "")
 }
 
+func holdMCPAuditLog(t *testing.T, path string) func() {
+	t.Helper()
+	f, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR, 0o600)
+	require.NoError(t, err)
+	release := func() {
+		if f != nil {
+			assert.NoError(t, f.Close())
+			f = nil
+		}
+	}
+	t.Cleanup(release)
+	require.NoError(t, unix.Flock(int(f.Fd()), unix.LOCK_EX|unix.LOCK_NB))
+	return release
+}
+
+func mcpChildEnv(auditPath string) []string {
+	childEnv := make([]string, 0, len(os.Environ())+3)
+	for _, entry := range os.Environ() {
+		if strings.HasPrefix(entry, tools.EnvAuditLog+"=") || strings.HasPrefix(entry, tools.EnvEnableRealPurchases+"=") {
+			continue
+		}
+		childEnv = append(childEnv, entry)
+	}
+	return append(childEnv,
+		runAsMCPServerEnv+"=1",
+		tools.EnvAuditLog+"="+auditPath,
+		tools.EnvEnableRealPurchases+"=",
+	)
+}
+
+func callMCPPreview(t *testing.T, session *gosdk.ClientSession) {
+	t.Helper()
+	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+	defer cancel()
+	result, err := session.CallTool(ctx, &gosdk.CallToolParams{
+		Name: "cudly_aws_ec2_ri_purchase",
+		Arguments: map[string]any{
+			"region":         "us-east-1",
+			"instance_type":  "m5.large",
+			"count":          1,
+			"term_years":     1,
+			"payment_option": "no-upfront",
+			"aws_profile":    "cudly-mcp-synthetic-profile",
+			"dry_run":        true,
+			"confirm":        false,
+		},
+	})
+	require.NoError(t, err)
+	require.False(t, result.IsError)
+	structured, err := json.Marshal(result.StructuredContent)
+	require.NoError(t, err)
+	var response tools.PurchaseResponse
+	require.NoError(t, json.Unmarshal(structured, &response))
+	require.True(t, response.Success)
+	require.True(t, response.DryRun)
+}
+
+func TestMainAuditLockTimeout(t *testing.T) {
+	t.Run("startup", func(t *testing.T) {
+		path := filepath.Join(t.TempDir(), "mcp-audit.jsonl")
+		original := []byte("existing-record\n")
+		require.NoError(t, os.WriteFile(path, original, 0o600))
+		holdMCPAuditLog(t, path)
+
+		exe, err := os.Executable()
+		require.NoError(t, err)
+		ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+		defer cancel()
+		cmd := exec.CommandContext(ctx, exe)
+		cmd.Env = mcpChildEnv(path)
+		var stderr bytes.Buffer
+		cmd.Stderr = &stderr
+		client := gosdk.NewClient(&gosdk.Implementation{Name: "test-client"}, nil)
+		session, err := client.Connect(ctx, &gosdk.CommandTransport{Command: cmd}, nil)
+		if session != nil {
+			defer func() { assert.NoError(t, session.Close()) }()
+		}
+		require.Error(t, err)
+		require.NoError(t, ctx.Err())
+		assert.Contains(t, stderr.String(), "timed out acquiring audit lock")
+		data, readErr := os.ReadFile(path)
+		require.NoError(t, readErr)
+		require.Equal(t, original, data)
+	})
+
+	t.Run("preview", func(t *testing.T) {
+		isolateFromAmbientAWS(t)
+		path := filepath.Join(t.TempDir(), "mcp-audit.jsonl")
+		original := []byte("existing-record\n")
+		require.NoError(t, os.WriteFile(path, original, 0o600))
+		exe, err := os.Executable()
+		require.NoError(t, err)
+
+		ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
+		defer cancel()
+		cmd := exec.CommandContext(ctx, exe)
+		cmd.Env = mcpChildEnv(path)
+		var stderr bytes.Buffer
+		cmd.Stderr = &stderr
+		client := gosdk.NewClient(&gosdk.Implementation{Name: "test-client"}, nil)
+		session, err := client.Connect(ctx, &gosdk.CommandTransport{Command: cmd}, nil)
+		if session != nil {
+			defer func() { assert.NoError(t, session.Close()) }()
+		}
+		require.NoError(t, err)
+		releaseHolder := holdMCPAuditLog(t, path)
+		callMCPPreview(t, session)
+		require.NoError(t, session.Close())
+		require.NoError(t, ctx.Err())
+		assert.Contains(t, stderr.String(), "timed out acquiring audit lock")
+		data, readErr := os.ReadFile(path)
+		require.NoError(t, readErr)
+		require.Equal(t, original, data)
+		releaseHolder()
+
+		freshCmd := exec.CommandContext(ctx, exe)
+		freshCmd.Env = mcpChildEnv(path)
+		var freshStderr bytes.Buffer
+		freshCmd.Stderr = &freshStderr
+		freshClient := gosdk.NewClient(&gosdk.Implementation{Name: "test-client"}, nil)
+		freshSession, err := freshClient.Connect(ctx, &gosdk.CommandTransport{Command: freshCmd}, nil)
+		if freshSession != nil {
+			defer func() { assert.NoError(t, freshSession.Close()) }()
+		}
+		require.NoError(t, err)
+		callMCPPreview(t, freshSession)
+		require.NoError(t, freshSession.Close())
+		require.NoError(t, ctx.Err())
+
+		data, readErr = os.ReadFile(path)
+		require.NoError(t, readErr)
+		require.True(t, bytes.HasPrefix(data, original))
+		lines := bytes.Split(bytes.TrimSuffix(data, []byte{'\n'}), []byte{'\n'})
+		require.Len(t, lines, 2)
+		var record common.AuditRecord
+		require.NoError(t, json.Unmarshal(lines[1], &record))
+		assert.Equal(t, "skipped", record.Status)
+		assert.True(t, record.DryRun)
+		assert.Equal(t, "cudly-mcp-synthetic-profile", record.CredentialScope)
+		assert.Equal(t, common.PurchaseSourceMCP, record.Source)
+		assert.Equal(t, common.ProviderAWS, record.Provider)
+		assert.Equal(t, string(common.ServiceEC2), record.Service)
+	})
+}
+
+func TestMainRejectsStdoutAuditLogBeforeProtocolTraffic(t *testing.T) {
+	info, err := os.Stat("/dev/stdout")
+	if err != nil {
+		t.Skipf("/dev/stdout unavailable: %v", err)
+	}
+	if info.Mode().IsRegular() {
+		t.Skip("/dev/stdout is a regular file in this environment")
+	}
+
+	exe, err := os.Executable()
+	require.NoError(t, err)
+
+	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+	defer cancel()
+
+	cmd := exec.CommandContext(ctx, exe)
+	childEnv := make([]string, 0, len(os.Environ())+2)
+	for _, entry := range os.Environ() {
+		if strings.HasPrefix(entry, tools.EnvAuditLog+"=") {
+			continue
+		}
+		childEnv = append(childEnv, entry)
+	}
+	cmd.Env = append(childEnv, runAsMCPServerEnv+"=1", tools.EnvAuditLog+"=/dev/stdout")
+	var stderr bytes.Buffer
+	cmd.Stderr = &stderr
+
+	client := gosdk.NewClient(&gosdk.Implementation{Name: "test-client"}, nil)
+	session, err := client.Connect(ctx, &gosdk.CommandTransport{
+		Command:           cmd,
+		TerminateDuration: time.Second,
+	}, nil)
+	if err == nil {
+		_, err = session.CallTool(ctx, &gosdk.CallToolParams{
+			Name: "cudly_aws_ec2_ri_purchase",
+			Arguments: map[string]any{
+				"region":         "us-east-1",
+				"instance_type":  "m5.large",
+				"count":          1,
+				"term_years":     1,
+				"payment_option": "no-upfront",
+			},
+		})
+		closeErr := session.Close()
+		if err == nil {
+			err = closeErr
+		}
+	}
+
+	require.Error(t, err)
+	assert.NotContains(t, err.Error(), "invalid message version tag")
+	assert.NoError(t, ctx.Err())
+	stderrText := stderr.String()
+	assert.Contains(t, stderrText, "failed to build server")
+	assert.Contains(t, stderrText, "non-regular audit log target")
+}
+
 // TestRealPurchasePastProviderRegistration is the regression guard for the
 // bug this file's blank imports fix: cudly-mcp never imported
 // providers/aws|azure|gcp, so their init()-registered factories were never

go.mod

@@ -64,7 +64,7 @@ require (
 	golang.org/x/net v0.58.0 // indirect
 	golang.org/x/oauth2 v0.36.0 // indirect
 	golang.org/x/sync v0.22.0 // indirect
-	golang.org/x/sys v0.47.0 // indirect
+	golang.org/x/sys v0.47.0
 	golang.org/x/text v0.41.0 // indirect
 	golang.org/x/time v0.15.0 // indirect
 	google.golang.org/api v0.274.0 // indirect
@@ -77,11 +77,12 @@ require (
 )
 
 require (
-	github.com/LeanerCloud/cloud-commitments-go/pkg v0.0.0-20260928074610-6168f8b5360d
-	github.com/LeanerCloud/cloud-commitments-go/providers/aws v0.0.0-20260928074610-6168f8b5360d
-	github.com/LeanerCloud/cloud-commitments-go/providers/azure v0.0.0-20260928074610-6168f8b5360d
-	github.com/LeanerCloud/cloud-commitments-go/providers/gcp v0.0.0-20260928074610-6168f8b5360d
+	github.com/LeanerCloud/cloud-commitments-go/pkg v0.0.0-20260928132534-fe940a89483d
+	github.com/LeanerCloud/cloud-commitments-go/providers/aws v0.0.0-20260928132534-fe940a89483d
+	github.com/LeanerCloud/cloud-commitments-go/providers/azure v0.0.0-20260928132534-fe940a89483d
+	github.com/LeanerCloud/cloud-commitments-go/providers/gcp v0.0.0-20260928132534-fe940a89483d
 	github.com/google/jsonschema-go v0.4.3
+	github.com/google/uuid v1.6.0
 	github.com/modelcontextprotocol/go-sdk v1.6.1
 )
 
@@ -106,7 +107,6 @@ require (
 	github.com/envoyproxy/go-control-plane/envoy v1.37.0 // indirect
 	github.com/envoyproxy/protoc-gen-validate v1.3.3 // indirect
 	github.com/go-jose/go-jose/v4 v4.1.4 // indirect
-	github.com/google/uuid v1.6.0 // indirect
 	github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
 	github.com/segmentio/asm v1.1.3 // indirect
 	github.com/segmentio/encoding v0.5.4 // indirect