Fix upsert() raising NOT NULL for omitted column on existing row (#878)

[agustin18]

Fixes #878.

Problem

When performing a partial .upsert() on an existing row where an omitted column is NOT NULL without a default value, SQLite raises sqlite3.IntegrityError: NOT NULL constraint failed: [table].[column].
This contradicts the documented guarantee:

"Any existing columns that are not referenced in the dictionary passed to .upsert() will be unchanged."

Root Cause

In modern SQLite (>=3.24) with native INSERT ... ON CONFLICT(pk) DO UPDATE, SQLite evaluates NOT NULL constraints on omitted columns during the INSERT clause evaluation before resolving the conflict on primary keys. When an omitted column is NOT NULL without a DEFAULT, SQLite aborts with an integrity error before reaching the DO UPDATE clause.

Fix

In Table.build_insert_queries_and_params():

1. Detect table columns that are notnull, have no default value, are not part of the primary key, and are omitted from all_columns (including checking caller-supplied not_null).
2. When such columns exist, include them in the INSERT column projection and supply a correlated scalar subquery for their values:

   (SELECT [col] FROM [table] WHERE [pk1] = ? AND [pk2] = ...)

   and bind the row's primary key parameters.
3. If the row exists, the subquery retrieves the current value, satisfying SQLite's NOT NULL check and triggering ON CONFLICT to execute DO UPDATE SET ... for caller-supplied columns only.
4. If the row does not exist, the subquery returns NULL, correctly preserving IntegrityError when attempting to insert new rows missing required columns.
5. Preserves full compatibility with tables using CHECK constraints, STRICT tables, and custom SQL conversions.

Tests

Added comprehensive test coverage to tests/test_upsert.py:

• test_upsert_omitted_not_null_column_on_existing_row: validates updating an existing row preserves omitted NOT NULL columns on both use_old_upsert=(False, True).
• test_upsert_all_omitted_not_null_compound_pk: validates batch upsert (batch_size=(1, 2)) with compound primary keys and multiple omitted NOT NULL columns.
• test_upsert_omitted_not_null_with_conversions_and_explicit_not_null: validates interactions with column conversions and explicit not_null arguments.
• test_upsert_omitted_not_null_with_check_constraint: validates tables with CHECK constraints on omitted columns.
• 100% test coverage across all added code paths.
• Verified with black, flake8, mypy, pyright, ty, and pytest --sqlite-autocommit.

---

📚 Documentation preview 📚: https://sqlite-utils--883.org.readthedocs.build/en/883/