Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,7 @@ core 镜像默认使用:

Usage 导入导出会使用 NDJSON 元数据记录保存模型价格、quota cache、监控设置、账号巡检调度和最近一次已结束的巡检结果快照,因此 WebDAV 备份恢复可以随 usage events 一起恢复监控相关状态。恢复的巡检快照用于迁移和问题追溯,默认只读;发起新的完整巡检后才允许重检、刷新令牌或执行账号变更。巡检日志不进入快照。监控日志保留会在每天服务器本地时间 02:00 自动清理,保存设置时也会立即清理一次;WebDAV 备份可单独设置保留天数,成功备份后会删除过期的 `usage-export-*.jsonl` 文件。

新导出的备份包含完整性 manifest。导入和 WebDAV 自动恢复默认拒绝无 manifest 的旧版备份;只有需要自动恢复某个可信旧备份时,才设置 `USAGE_ALLOW_LEGACY_RESTORE=true`
新导出的备份包含完整性 manifest。管理 API 和页面默认拒绝或要求确认无 manifest 的旧版备份;Docker WebDAV 自动恢复在过渡阶段会强制启用旧版兼容导入,带 manifest 的新备份仍会严格校验完整性

建议在生产环境中为该目录配置持久化 volume。

Expand All @@ -322,7 +322,6 @@ WEBDAV_URL
WEBDAV_USERNAME
WEBDAV_PASSWORD
MANAGEMENT_PASSWORD
USAGE_ALLOW_LEGACY_RESTORE
```

### 账号巡检
Expand Down
3 changes: 1 addition & 2 deletions README_EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,7 @@ It stores:

Usage export/import uses NDJSON metadata records for model prices, quota cache, monitoring settings, the account-inspection schedule, and the latest finished inspection-result snapshot, so WebDAV backup restore can recover the monitoring-related state together with usage events. Restored inspection snapshots are read-only for migration and troubleshooting; a new full inspection must run before rechecking accounts, refreshing tokens, or changing account state. Inspection logs are not included. Monitoring log retention runs daily at 02:00 server local time and also runs once immediately when settings are saved; WebDAV backups can use separate retention days, deleting expired `usage-export-*.jsonl` files after successful backups.

New exports include an integrity manifest. Import and automatic WebDAV restore reject manifest-free legacy backups by default; set `USAGE_ALLOW_LEGACY_RESTORE=true` only when one trusted legacy backup must be restored automatically.
New exports include an integrity manifest. Management API and UI imports reject or require confirmation for manifest-free legacy backups; during the compatibility transition, Docker WebDAV restore force-enables legacy import while continuing to verify manifest-backed backups strictly.

Configure a persistent volume for this directory in production.

Expand All @@ -294,7 +294,6 @@ WEBDAV_URL
WEBDAV_USERNAME
WEBDAV_PASSWORD
MANAGEMENT_PASSWORD
USAGE_ALLOW_LEGACY_RESTORE
```

### Account inspection
Expand Down
3 changes: 1 addition & 2 deletions cliproxyapi-pro-core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,6 @@ Release workflow 会从 Core、models 和定制层三个不可变提交中取最
- `WEBDAV_USERNAME`
- `WEBDAV_PASSWORD`
- `MANAGEMENT_PASSWORD`
- `USAGE_ALLOW_LEGACY_RESTORE` — 可选,默认 `false`;仅在需要自动导入可信的无 manifest 旧备份时设为 `true`。

恢复文件查找同时支持:

Expand All @@ -282,7 +281,7 @@ usage-export-YYYYMMDD_HHMMSS.json
usage-export-YYYYMMDD_HHMMSS.jsonl
```

自动恢复默认只导入带 manifest 的备份。无 manifest 文件会被跳过;只有设置 `USAGE_ALLOW_LEGACY_RESTORE=true` 时,entrypoint 才会调用 `/usage/import?allow_legacy=1`,并在日志中明确记录正在使用未经完整性校验的兼容路径。
Docker WebDAV 自动恢复在过渡阶段固定调用 `/usage/import?allow_legacy=1`。带 manifest 的新备份仍会严格校验完整性;无 manifest 的旧版备份会强制导入,并在日志中明确记录正在使用未经完整性校验的兼容路径。管理 API 的普通导入仍默认拒绝无 manifest 文件

导入请求使用:

Expand Down
3 changes: 1 addition & 2 deletions cliproxyapi-pro-core/README_EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,6 @@ When all variables below are configured, `entrypoint.sh` waits for the local API
- `WEBDAV_USERNAME`
- `WEBDAV_PASSWORD`
- `MANAGEMENT_PASSWORD`
- `USAGE_ALLOW_LEGACY_RESTORE` — optional, default `false`; set to `true` only to automatically import a trusted manifest-free legacy backup.

Restore lookup supports both backup names:

Expand All @@ -267,7 +266,7 @@ usage-export-YYYYMMDD_HHMMSS.json
usage-export-YYYYMMDD_HHMMSS.jsonl
```

Automatic restore imports manifest-backed backups by default. A manifest-free backup is skipped unless `USAGE_ALLOW_LEGACY_RESTORE=true`; when enabled, the entrypoint calls `/usage/import?allow_legacy=1` and logs that the unverified compatibility path is in use.
During the compatibility transition, Docker WebDAV restore always calls `/usage/import?allow_legacy=1`. Manifest-backed backups are still verified strictly; manifest-free legacy backups are imported with an explicit warning that integrity cannot be verified. Normal management API imports still reject manifest-free files by default.

The import request uses:

Expand Down
12 changes: 8 additions & 4 deletions cliproxyapi-pro-core/embeddedusage/pricing.go
Original file line number Diff line number Diff line change
Expand Up @@ -377,20 +377,24 @@ func (s *Store) UpsertModelPriceRule(ctx context.Context, rule ModelPriceRule, a
}
defer func() { _ = tx.Rollback() }()

var currentVersion int
var currentLocked int
var currentRaw string
err = tx.QueryRowContext(ctx, `select a.active_version, a.locked, v.rule_json
err = tx.QueryRowContext(ctx, `select a.locked, v.rule_json
from model_price_rules a join model_price_rule_versions v
on v.provider = a.provider and v.model = a.model and v.version = a.active_version
where a.provider = ? and a.model = ?`, rule.Provider, rule.Model).Scan(&currentVersion, &currentLocked, &currentRaw)
where a.provider = ? and a.model = ?`, rule.Provider, rule.Model).Scan(&currentLocked, &currentRaw)
if err != nil && err != sql.ErrNoRows {
return ModelPriceRule{}, false, err
}
if currentLocked != 0 && rule.Source == modelPriceSourceModelsDev && !allowLockedOverride {
return ModelPriceRule{}, false, nil
}
rule.Version = currentVersion + 1
var latestVersion int
if err := tx.QueryRowContext(ctx, `select coalesce(max(version), 0) from model_price_rule_versions where provider = ? and model = ?`,
rule.Provider, rule.Model).Scan(&latestVersion); err != nil {
return ModelPriceRule{}, false, err
}
rule.Version = latestVersion + 1
rule.ID = 0
rule.UpdatedAt = time.Now().UnixMilli()
raw, err := json.Marshal(rule)
Expand Down
38 changes: 38 additions & 0 deletions cliproxyapi-pro-core/embeddedusage/server_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,44 @@ func TestUsageImportRejectsTamperedManifestBackupBeforeWriting(t *testing.T) {
}
}

func TestUsageImportRestoresModelPriceRuleWhenOnlyNewerHistoryRemains(t *testing.T) {
ctx := context.Background()
sourceStore := openTestStore(t)
sourceRule := testGPT56PriceRule()
sourceRule.Base.Input = 1.25
if _, changed, err := sourceStore.UpsertModelPriceRule(ctx, sourceRule, true); err != nil || !changed {
t.Fatalf("source UpsertModelPriceRule() = changed:%v err:%v", changed, err)
}
exportRecorder := httptest.NewRecorder()
testUsageRouter(sourceStore).ServeHTTP(exportRecorder, httptest.NewRequest(http.MethodGet, "/usage/export", nil))
if exportRecorder.Code != http.StatusOK {
t.Fatalf("export status = %d; body=%s", exportRecorder.Code, exportRecorder.Body.String())
}

targetStore := openTestStore(t)
targetRule := testGPT56PriceRule()
if _, changed, err := targetStore.UpsertModelPriceRule(ctx, targetRule, true); err != nil || !changed {
t.Fatalf("target first UpsertModelPriceRule() = changed:%v err:%v", changed, err)
}
targetRule.Base.Input = 9.99
if _, changed, err := targetStore.UpsertModelPriceRule(ctx, targetRule, true); err != nil || !changed {
t.Fatalf("target second UpsertModelPriceRule() = changed:%v err:%v", changed, err)
}
if err := targetStore.DeleteModelPriceRule(ctx, "", targetRule.Model); err != nil {
t.Fatalf("DeleteModelPriceRule() error = %v", err)
}

importRecorder := httptest.NewRecorder()
testUsageRouter(targetStore).ServeHTTP(importRecorder, httptest.NewRequest(http.MethodPost, "/usage/import", bytes.NewReader(exportRecorder.Body.Bytes())))
if importRecorder.Code != http.StatusOK {
t.Fatalf("import status = %d, want 200; body=%s", importRecorder.Code, importRecorder.Body.String())
}
rules, err := targetStore.ActiveModelPriceRules(ctx)
if err != nil || len(rules) != 1 || rules[0].Model != sourceRule.Model || rules[0].Base.Input != sourceRule.Base.Input || rules[0].Version <= 2 {
t.Fatalf("restored rules = %+v err:%v; want imported rule newer than retained version 2", rules, err)
}
}

func TestHandleUsageResetClearsStatisticsAndReturnsGeneration(t *testing.T) {
store := openTestStore(t)
insertTestUsageEvents(t, store,
Expand Down
34 changes: 9 additions & 25 deletions cliproxyapi-pro-core/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,6 @@ log() {
echo "$(date '+%Y-%m-%d %H:%M:%S') [$1] [$2] $3"
}

is_truthy() {
case "$(printf '%s' "$1" | tr '[:upper:]' '[:lower:]')" in
1|true|yes|on) return 0 ;;
*) return 1 ;;
esac
}

# ==========================================
# 环境变量配置
# ==========================================
Expand All @@ -26,7 +19,6 @@ WEBDAV_URL="${WEBDAV_URL:-}"
WEBDAV_USERNAME="${WEBDAV_USERNAME:-}"
WEBDAV_PASSWORD="${WEBDAV_PASSWORD:-}"
MANAGEMENT_PASSWORD="${MANAGEMENT_PASSWORD:-}"
USAGE_ALLOW_LEGACY_RESTORE="${USAGE_ALLOW_LEGACY_RESTORE:-false}"

# ==========================================
# 1. 启动 komari-agent
Expand Down Expand Up @@ -75,26 +67,18 @@ if [ -n "$WEBDAV_URL" ] && [ -n "$WEBDAV_USERNAME" ] && [ -n "$WEBDAV_PASSWORD"
"$WEBDAV_URL/$LATEST_FILE" -o /tmp/usage-restore.jsonl

if [ -f /tmp/usage-restore.jsonl ]; then
IMPORT_URL="http://127.0.0.1:8317/v0/management/usage/import"
IMPORT_URL="http://127.0.0.1:8317/v0/management/usage/import?allow_legacy=1"
if ! awk 'NF { print; exit }' /tmp/usage-restore.jsonl | \
grep -Eq '"record_type"[[:space:]]*:[[:space:]]*"backup_manifest"'; then
if is_truthy "$USAGE_ALLOW_LEGACY_RESTORE"; then
IMPORT_URL="$IMPORT_URL?allow_legacy=1"
log "UsageRestore" "WARN" "Importing manifest-free legacy backup because USAGE_ALLOW_LEGACY_RESTORE is enabled."
else
IMPORT_URL=""
log "UsageRestore" "WARN" "Backup has no integrity manifest; skipping automatic restore. Set USAGE_ALLOW_LEGACY_RESTORE=true only for a trusted legacy backup."
fi
fi
if [ -n "$IMPORT_URL" ]; then
log "UsageRestore" "INFO" "Importing usage data..."
RESULT=$(curl -s -X POST \
-H "Content-Type: application/x-ndjson" \
-H "Authorization: Bearer $MANAGEMENT_PASSWORD" \
--data-binary @/tmp/usage-restore.jsonl \
"$IMPORT_URL")
log "UsageRestore" "INFO" "Import result: $RESULT"
log "UsageRestore" "WARN" "Importing manifest-free legacy backup without integrity verification during the compatibility transition."
fi
log "UsageRestore" "INFO" "Importing usage data..."
RESULT=$(curl -s -X POST \
-H "Content-Type: application/x-ndjson" \
-H "Authorization: Bearer $MANAGEMENT_PASSWORD" \
--data-binary @/tmp/usage-restore.jsonl \
"$IMPORT_URL")
log "UsageRestore" "INFO" "Import result: $RESULT"
rm -f /tmp/usage-restore.jsonl
else
log "UsageRestore" "WARN" "Download failed."
Expand Down
6 changes: 3 additions & 3 deletions docs/code-review/2026-07.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ The pinned PR baseline lives in `compatibility/upstream.env`. Pull requests and
| STATE-005 | P1 | Runtime restore transaction | Routing cursors and auth runtime statistics were restored in separate transactions | A stats failure could leave restored cursor ordering paired with old selection counters | Force the auth-runtime insert after the cursor import succeeds | Restore both runtime record types in one SQLite transaction, then apply the committed snapshot to the live manager | Runtime import rollback and live restore tests | Resolved |
| STATE-006 | P2 | Event deduplication | Generated `event_hash` values omitted `api_key_hash` | Otherwise identical hashless events from different client API keys could collide and lose attribution | Normalize matching payloads that differ only by API key | Include the hashed API-key identity in newly generated event hashes while preserving hashes from older exports | Internal usage hash test | Resolved |
| STATE-007 | P2 | SQLite export consistency | Export queried events, pricing, quota, routing, auth-runtime state, and settings independently | A write between queries could produce a valid manifest for records representing different database instants | Pause export after reading events and attempt an adjacent settings write | Read every SQLite-backed export record through one read transaction and stamp them with one `exported_at_ms` | `TestUsageExportSnapshotKeepsAdjacentWritesOutOfEveryRecordType` and embeddedusage race detector | Resolved |
| STATE-008 | P2 | Legacy backup trust | Manifest-free backups were accepted by the API, management UI, and automatic WebDAV restore without an explicit trust decision | Truncated or modified legacy data could be restored as though it had passed integrity verification | Import an event-only JSONL without a manifest or place it as the latest WebDAV backup | Reject manifest-free imports by default, require a confirmation or explicit API compatibility flag, and keep automatic legacy restore behind `USAGE_ALLOW_LEGACY_RESTORE` | API default-rejection/explicit-opt-in tests, management compatibility-gate tests, and entrypoint syntax validation | Resolved |
| STATE-008 | P2 | Legacy backup trust | Manifest-free backups were accepted by the API, management UI, and automatic WebDAV restore without an explicit trust decision | Truncated or modified legacy data could be restored as though it had passed integrity verification | Import an event-only JSONL without a manifest or place it as the latest WebDAV backup | Reject manifest-free API imports by default and require UI confirmation; during the Docker transition, force-enable automatic legacy restore with an explicit warning | API default-rejection/explicit-opt-in tests, management compatibility-gate tests, and entrypoint syntax validation | Resolved with temporary Docker compatibility exception |
| ACTION-001 | P1 | Account inspection actions | Manual action, single-account recheck, and token-refresh requests trusted client `key`, `fileName`, and `authIndex` combinations independently | A stale or crafted request could execute against a different live auth than the displayed inspection result | Submit one result's key with another auth's index and file name | Bind every request to the server-side current snapshot, canonicalize identity fields, revalidate the live auth before mutation, and serialize action execution | Snapshot binding and stale-runtime identity tests | Resolved |
| ACTION-002 | P1 | Plugin virtual deletion | Deletion used the result file name even when multiple plugin virtual auths shared one physical source | An invalid virtual project could delete the credential source for every project expanded from that file | Register two virtual auths with one source and delete the source-facing result | Reject account-inspection deletion while more than one live plugin virtual auth references the source | Shared-source deletion regression test | Resolved |
| ACTION-003 | P1 | Plugin virtual error state | Inspection error writeback applied one mutation to every runtime auth sharing the same source path | A failure in one virtual project marked healthy sibling projects unavailable and persisted an ambiguous shared error | Write an inspection error for the second identity in a two-project source | Keep identity-local inspection errors on the targeted runtime auth; reserve shared-source persistence for shared status such as disabled state | Plugin virtual error-isolation test | Resolved |
Expand Down Expand Up @@ -173,7 +173,7 @@ The pinned PR baseline lives in `compatibility/upstream.env`. Pull requests and
## Package 7 Residual Risks

- SQLite-backed records now come from one read transaction, but account-inspection schedule/snapshot files are still read and restored through external callbacks. SQLite cannot share a transaction with these files, so the mixed backup and restore remain staged and idempotent rather than cross-resource atomic.
- Manifest-free compatibility remains intentionally available for trusted historical backups. The API, management UI, and entrypoint now require explicit opt-in and surface the lack of verification, but they cannot prove integrity for a legacy file that never contained a manifest.
- Manifest-free compatibility remains intentionally available for historical backups. The API and management UI require explicit opt-in; Docker WebDAV restore temporarily opts in by policy and logs the lack of verification. No path can prove integrity for a legacy file that never contained a manifest.
- Usage payloads still originate in an in-process upstream queue. Retry and graceful-shutdown paths are covered, but an abrupt process or host crash before SQLite persistence can still lose that in-memory window.

## Package 7 Validation Results
Expand All @@ -182,7 +182,7 @@ The pinned PR baseline lives in `compatibility/upstream.env`. Pull requests and
- Core `v7.2.94`: guarded-source preflight, rejected reapplication invariance, embedded usage tests, management handlers, plugin host/store, Redis queue, auth scheduler tests, and the server build passed on a fresh generated checkout.
- The embedded usage packages passed the Go race detector. Regression coverage includes a single SQLite export snapshot across all database-backed record types and default rejection plus explicit opt-in for manifest-free imports.
- Management `v1.18.5`: overlay collision preflight, repeated-application invariance, 89 Bun tests, ESLint with no warnings, TypeScript checking, and the production single-file build passed on a fresh checkout.
- Frontend coverage verifies that only a first non-empty `backup_manifest` record bypasses the legacy confirmation gate; WebDAV entrypoint syntax validation covers the explicit `USAGE_ALLOW_LEGACY_RESTORE` compatibility path.
- Frontend coverage verifies that only a first non-empty `backup_manifest` record bypasses the legacy confirmation gate; WebDAV entrypoint syntax validation covers the forced Docker compatibility path.

## Package 6 Residual Risks

Expand Down
Loading