Skip to content

[CPBench] Benchmark results Google Sheets writer#1182

Open
fbonc wants to merge 3 commits into
sunlabuiuc:masterfrom
fbonc:cpbench-runner
Open

[CPBench] Benchmark results Google Sheets writer#1182
fbonc wants to merge 3 commits into
sunlabuiuc:masterfrom
fbonc:cpbench-runner

Conversation

@fbonc

@fbonc fbonc commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Contributor: Felipe Amaral Bonchristiano (felipea5@illinois.edu)
Contribution Type: New benchmarks/ tool

Description
Adds benchmarks/cpbench/writers, a small CLI for pushing CPBench result JSON files to a Google Sheet so runs accumulate in one place instead of living as scattered local files.

Files to review

  • parse_json.py: load_json + flatten, which split a result file into its metadata, aggregated, and per_seed blocks as (header, rows) pairs. Per-seed rows from every seed are concatenated into one block; each row carries its own seed column.
  • sheets_writer.py: SheetsWriter.append(worksheet_name, rows, columns). Creates the worksheet if absent, merges new columns into the existing header, and appends.
  • cli.py: argparse entrypoint plus the terminal preview rendering.
  • __init__.py: package stuff

Usage

# preview: no credentials, no sheet id, nothing written
python -m benchmarks.cpbench.writers.cli --results results/run.json --preview

# append to a sheet
python -m benchmarks.cpbench.writers.cli --results results/run.json \
    --sheet-id 1AbC...xyz --credentials service_account.json

Aggregated and per-seed rows go to separate worksheets, defaulting to "Aggregated Worksheet" and "Per-seed Results" and overridable with --aggregated-worksheet / --per-seed-worksheet. --sheet-id and --credentials are required only when not previewing.

Notes

  • Every written row is prefixed with the run's metadata (dataset, task, model, cp_method, timestamp, protocol), so a row identifies itself once many runs share a worksheet.
  • Non-scalar metadata is flattened to text; numbers stay numeric.
  • Rows are aligned to the sheet's header by column name before appending. New columns are appended, and missing columns are written blank.
  • gspread is imported lazily inside _open_worksheet, so it is only needed to actually write — --preview and importing the package work without it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant