Skip to content

Transaction timeout during checksum computation for large filepath fields #1386

Closed

Description

@dimitri-yatsenko

Problem

In three-part make() methods, the transaction can expire while checksums are computed for large files with filepath fields. This causes the entire make() to fail, not just a performance degradation.

The checksum computation for large files takes long enough that the database transaction times out before the insert can complete.

Context

This affects the 0.14.x branch. The issue occurs when:

  • Working with large files (videos, imaging data, etc.)
  • Using filepath attributes that reference these large files
  • The checksum computation happens within the transaction
  • Computation time exceeds the transaction timeout

Impact

  • make() fails entirely for large files
  • Data that was already computed is lost
  • Users cannot reliably use filepath fields with large files

Branch

maint/0.14

Note

This will be addressed differently in DataJoint 2.0+.

Activity

  1. changed the title [-]Slow inserts due to checksum computation for large filepath fields[/-] [+]Transaction timeout during checksum computation for large filepath fields[/+] on Feb 5, 2026
  2. added 3 commits that reference this issue on Feb 5, 2026
    446277e
    83ade69
    6d02652
  3. dimitri-yatsenko commented on Feb 5, 2026

    @dimitri-yatsenko
    MemberAuthor

    Fixed in v0.14.8 with the new filepath_checksum_size_limit_insert config option. See PR #1387.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions