upsert_all() overwrites omitted columns with NULL when records in the same batch contain different fields. This reproduces on current main (6bc1d33d583c54bd69fbdd2071117e2d38c354a1): the batch-wide column list fills each record's missing fields with None, and the generated ON CONFLICT ... DO UPDATE assigns those values to the existing rows.
from sqlite_utils import Database
db = Database(memory=True)
table = db.table("dogs")
table.insert_all([
{"id": 1, "name": "Cleo", "age": 5},
{"id": 2, "name": "Nixie", "age": 6},
], pk="id")
table.upsert_all([
{"id": 1, "age": 7},
{"id": 2, "name": "Nixie II"},
])
print(list(table.rows_where(order_by="id")))
Expected:
[{'id': 1, 'name': 'Cleo', 'age': 7},
{'id': 2, 'name': 'Nixie II', 'age': 6}]
Actual:
[{'id': 1, 'name': None, 'age': 7},
{'id': 2, 'name': 'Nixie II', 'age': None}]
The upsert documentation says columns omitted from an upsert dictionary remain unchanged, and describes upsert_all() as performing upserts.
The relevant paths are missing-field expansion and the conflict-update assignments. The update columns need to reflect the fields supplied by each record while preserving record order.
Verified with Python 3.13.12 / SQLite 3.53.0. The existing tests/test_upsert.py suite passes (18 tests).
Found and reproduced using OpenAI Codex.
upsert_all()overwrites omitted columns withNULLwhen records in the same batch contain different fields. This reproduces on current main (6bc1d33d583c54bd69fbdd2071117e2d38c354a1): the batch-wide column list fills each record's missing fields withNone, and the generatedON CONFLICT ... DO UPDATEassigns those values to the existing rows.Expected:
[{'id': 1, 'name': 'Cleo', 'age': 7}, {'id': 2, 'name': 'Nixie II', 'age': 6}]Actual:
[{'id': 1, 'name': None, 'age': 7}, {'id': 2, 'name': 'Nixie II', 'age': None}]The upsert documentation says columns omitted from an upsert dictionary remain unchanged, and describes
upsert_all()as performing upserts.The relevant paths are missing-field expansion and the conflict-update assignments. The update columns need to reflect the fields supplied by each record while preserving record order.
Verified with Python 3.13.12 / SQLite 3.53.0. The existing
tests/test_upsert.pysuite passes (18 tests).Found and reproduced using OpenAI Codex.