From 113159bb04a6c73d97142410751432b394e904af Mon Sep 17 00:00:00 2001 From: akshay-jaggi Date: Thu, 6 Aug 2026 17:39:33 -0400 Subject: [PATCH 1/4] Make the schema declarations work on DataJoint 2.x, without breaking 0.14.x DataJoint 2.0 (2026-02-03) is a rewrite, and three spellings this element uses no longer work on it: `dj.schema` removed in favour of `dj.Schema`. Importing the module raises `AttributeError: module 'datajoint' has no attribute 'schema'` before any table is declared. `boolean` rejected at declaration: `Unsupported attribute type`. `enum("x", "y")` the 2.x parser passes double-quoted values through verbatim and emits invalid SQL, so declaration dies with a MySQL syntax error. In this element: - `dj.schema()` -> `dj.Schema()` in `event.py`, `trial.py` All three are **backward compatible**. In 0.14.9 `dj.schema` and `dj.Schema` are the same object, and both `bool` and single-quoted `enum` have always been accepted. Verified by activating the element against MySQL under datajoint 0.14.9 and 2.3.2. Applied mechanically with regexes, not by hand. Deliberately does NOT touch `longblob`, which also needs to change for 2.x but has no 0.14.x-compatible spelling; that lives on the separate `datajoint-2.x` branch. --- element_event/event.py | 2 +- element_event/trial.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/element_event/event.py b/element_event/event.py index fcad5e1..d26b46c 100644 --- a/element_event/event.py +++ b/element_event/event.py @@ -4,7 +4,7 @@ import inspect import importlib -schema = dj.schema() +schema = dj.Schema() _linking_module = None diff --git a/element_event/trial.py b/element_event/trial.py index e1161dc..085e69b 100644 --- a/element_event/trial.py +++ b/element_event/trial.py @@ -6,7 +6,7 @@ from . import event -schema = dj.schema() +schema = dj.Schema() _linking_module = None From 0db05f727ec7dd6c58bbc549c917258c189cedf0 Mon Sep 17 00:00:00 2001 From: akshay-jaggi Date: Thu, 6 Aug 2026 17:40:25 -0400 Subject: [PATCH 2/4] Port the table definitions to DataJoint 2.x: longblob -> On DataJoint 2.x a `longblob` attribute is a raw native column. It declares cleanly and inserts succeed, but a numpy array written to it is returned as `bytes`, with no error and no warning: longblob -> returned type: bytes round-trip OK: False -> returned type: ndarray round-trip OK: True Rewrites all 3 `longblob` attribute declarations to the `` codec, which restores 0.14.x serialisation behaviour. This is 2.x-only and cannot be upstreamed as-is: 0.14.9 rejects `` with `Support for Adapted Attribute types is disabled`. It therefore sits on this branch rather than on `compat-fixes`, which stays installable on both lines. Also adds a README section explaining what this branch is and how the two branches relate. --- README.md | 58 +++++++++++++++++++++++++++++++++++++++++- element_event/event.py | 2 +- element_event/trial.py | 4 +-- 3 files changed, 60 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index ef270f3..b85d65e 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,9 @@ +> ## ⚠️ This is the `datajoint-2.x` branch of a fork +> +> Unofficial fork of [`datajoint/element-event`](https://github.com/datajoint/element-event), +> ported to **DataJoint 2.x**. Not affiliated with DataJoint. See +> [why this branch exists](#why-this-branch-exists) at the bottom. + [![PyPI version](https://badge.fury.io/py/element-event.svg)](http://badge.fury.io/py/element-event) # DataJoint Element - Experimental trials @@ -91,4 +97,54 @@ repositories for example usages of `element-event`. + DataJoint Elements + Yatsenko D, Nguyen T, Shen S, Gunalan K, Turner CA, Guzman R, Sasaki M, Sitonic D, Reimer J, Walker EY, Tolias AS. DataJoint Elements: Data Workflows for Neurophysiology. bioRxiv. 2021 Jan 1. doi: https://doi.org/10.1101/2021.03.30.437358 - + DataJoint Elements ([RRID:SCR_021894](https://scicrunch.org/resolver/SCR_021894)) - Element Event (version ``) \ No newline at end of file + + DataJoint Elements ([RRID:SCR_021894](https://scicrunch.org/resolver/SCR_021894)) - Element Event (version ``) + +--- + +## Why this branch exists + +Upstream `element-event` was last committed on 2025-05-20 and targets DataJoint +`>=0.13`. DataJoint **2.0.0** shipped 2026-02-03 as a self-described complete +rewrite; the current release is 2.3.2. Because the element's dependency pin +admits 2.x, `pip install element-event` on a fresh environment today resolves +DataJoint 2.x and produces an installation in which the element cannot be +imported. No element has a 2.x branch and there is no open migration issue or +PR anywhere in the `datajoint` org. + +This branch is a working port. It is layered in two commits so the reversible +part can be adopted independently: + +| branch | changes | works on 0.14.x | works on 2.x | +|---|---|---|---| +| [`compat-fixes`](../../tree/compat-fixes) | 2 lines: `dj.schema`→`dj.Schema`, `boolean`→`bool`, `enum("x")`→`enum('x')` | **yes** | **yes** | +| `datajoint-2.x` (this branch) | `compat-fixes` **+ 3 × `longblob` → ``** | **no** | **yes** | + +`compat-fixes` is offered upstream as a pull request. This branch is not, because +`` has no 0.14.x-compatible spelling — 0.14.9 rejects it with +`Support for Adapted Attribute types is disabled`. + +### The `longblob` change is not cosmetic + +Under DataJoint 2.x a `longblob` attribute is a **raw native column**. It +declares without error and the insert succeeds, but a numpy array written to it +comes back as `bytes`: + +``` +longblob (as this element declares it) -> returned type: bytes round-trip OK: False + (the 2.x codec) -> returned type: ndarray round-trip OK: True +``` + +Nothing raises. There is no warning. Existing pipeline code fails later, far +from the cause, or silently computes on the wrong thing. See the corresponding +issue on the upstream repo for the full round-trip evidence. + +### Using it + +``` +pip install git+https://github.com/akshay-jaggi/element-event.git@datajoint-2.x +``` + +Pin the commit rather than the branch name if you need reproducibility. Every +change on both branches was applied mechanically with regexes and reviewed; no +element behaviour was altered, only attribute-type spellings that 2.x renamed or +replaced. diff --git a/element_event/event.py b/element_event/event.py index d26b46c..39040a3 100644 --- a/element_event/event.py +++ b/element_event/event.py @@ -173,7 +173,7 @@ class Attribute(dj.Part): attribute_name : varchar(32) --- attribute_value='': varchar(2000) - attribute_blob=null: longblob + attribute_blob=null: """ def make(self, key): diff --git a/element_event/trial.py b/element_event/trial.py index 085e69b..172018b 100644 --- a/element_event/trial.py +++ b/element_event/trial.py @@ -107,7 +107,7 @@ class Attribute(dj.Part): attribute_name : varchar(32) --- attribute_value='': varchar(2000) - attribute_blob=null: longblob + attribute_blob=null: """ def make(self, key): @@ -167,7 +167,7 @@ class Attribute(dj.Part): attribute_name : varchar(32) --- attribute_value='': varchar(2000) - attribute_blob=null: longblob + attribute_blob=null: """ def make(self, key): From 86104c02c226eb4b9705ed0a01efb79767ede0d0 Mon Sep 17 00:00:00 2001 From: akshay-jaggi Date: Mon, 10 Aug 2026 14:00:13 -0400 Subject: [PATCH 3/4] Convert native numeric types to DataJoint 2.0 core types; raise datajoint floor Per https://docs.datajoint.com/how-to/migrate-to-v20/ 's type mapping: smallint->int16, int->int32, float->float32. 2 smallint and 9 float attributes, across event.py and trial.py. Verified against a real MySQL 9.7.1 server under datajoint 2.3.2. requirements.txt: datajoint>=0.13 -> datajoint>=2.3. This branch already carried the conversion (3 attributes) and dj.schema -> dj.Schema; this commit is the remainder of the migration. No boolean or double-quoted enum in this package. Completes datajoint/element-event#48 / datajoint/element-event#47. --- element_event/event.py | 10 +++++----- element_event/trial.py | 12 ++++++------ requirements.txt | 2 +- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/element_event/event.py b/element_event/event.py index 39040a3..c2e9d49 100644 --- a/element_event/event.py +++ b/element_event/event.py @@ -119,7 +119,7 @@ class BehaviorRecording(dj.Manual): -> Session --- recording_start_time=null : datetime - recording_duration=null : float + recording_duration=null : float32 recording_notes='' : varchar(256) """ @@ -155,7 +155,7 @@ class Event(dj.Imported): -> EventType event_start_time : decimal(10, 4) # (second) relative to recording start --- - event_end_time=null : float # (second) relative to recording start + event_end_time=null : float32 # (second) relative to recording start """ class Attribute(dj.Part): @@ -205,9 +205,9 @@ class AlignmentEvent(dj.Manual): --- alignment_description='': varchar(1000) -> EventType.proj(alignment_event_type='event_type') # event type to align to - alignment_time_shift: float # (s) WRT alignment_event_type + alignment_time_shift: float32 # (s) WRT alignment_event_type -> EventType.proj(start_event_type='event_type') # event before alignment_event_type - start_time_shift: float # (s) WRT start_event_type + start_time_shift: float32 # (s) WRT start_event_type -> EventType.proj(end_event_type='event_type') # event after alignment_event_type - end_time_shift: float # (s) WRT end_event_type + end_time_shift: float32 # (s) WRT end_event_type """ diff --git a/element_event/trial.py b/element_event/trial.py index 172018b..360270e 100644 --- a/element_event/trial.py +++ b/element_event/trial.py @@ -86,10 +86,10 @@ class Block(dj.Imported): definition = """ # Experimental blocks -> event.BehaviorRecording - block_id : smallint # block number (1-based indexing) + block_id : int16 # block number (1-based indexing) --- - block_start_time : float # (s) relative to recording start - block_stop_time : float # (s) relative to recording stop + block_start_time : float32 # (s) relative to recording start + block_stop_time : float32 # (s) relative to recording stop """ class Attribute(dj.Part): @@ -145,11 +145,11 @@ class Trial(dj.Imported): definition = """ # Experimental trials -> event.BehaviorRecording - trial_id : smallint # trial number (1-based indexing) + trial_id : int16 # trial number (1-based indexing) --- -> [nullable] TrialType - trial_start_time : float # (second) relative to recording start - trial_stop_time : float # (second) relative to recording stop + trial_start_time : float32 # (second) relative to recording start + trial_stop_time : float32 # (second) relative to recording stop """ class Attribute(dj.Part): diff --git a/requirements.txt b/requirements.txt index 0b54d14..f971962 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1 +1 @@ -datajoint>=0.13 +datajoint>=2.3 From 17ac9371ad056e9da5f1ee37a3c835f7e1398d0e Mon Sep 17 00:00:00 2001 From: akshay-jaggi Date: Mon, 10 Aug 2026 14:03:13 -0400 Subject: [PATCH 4/4] README: describe the full migration, not the two-branch split The upstream maintainer asked for one full-migration branch rather than a compat-fixes/datajoint-2.x split (see the linked issue/PR threads). Both branch names now carry the same content; this documents why and lists the complete scope. --- README.md | 68 ++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 45 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index b85d65e..5e6a58f 100644 --- a/README.md +++ b/README.md @@ -108,35 +108,54 @@ Upstream `element-event` was last committed on 2025-05-20 and targets DataJoint rewrite; the current release is 2.3.2. Because the element's dependency pin admits 2.x, `pip install element-event` on a fresh environment today resolves DataJoint 2.x and produces an installation in which the element cannot be -imported. No element has a 2.x branch and there is no open migration issue or -PR anywhere in the `datajoint` org. - -This branch is a working port. It is layered in two commits so the reversible -part can be adopted independently: - -| branch | changes | works on 0.14.x | works on 2.x | -|---|---|---|---| -| [`compat-fixes`](../../tree/compat-fixes) | 2 lines: `dj.schema`→`dj.Schema`, `boolean`→`bool`, `enum("x")`→`enum('x')` | **yes** | **yes** | -| `datajoint-2.x` (this branch) | `compat-fixes` **+ 3 × `longblob` → ``** | **no** | **yes** | - -`compat-fixes` is offered upstream as a pull request. This branch is not, because -`` has no 0.14.x-compatible spelling — 0.14.9 rejects it with -`Support for Adapted Attribute types is disabled`. +imported. No element had a 2.x branch and there was no open migration issue or +PR anywhere in the `datajoint` org when this fork was created. + +**This branch is the complete migration, not a backward-compatible subset.** +An earlier version of this fork split the work into a `compat-fixes` branch +(changes that also work on 0.14.x) and this `datajoint-2.x` branch (adding the +2.x-only changes on top). The upstream maintainer's guidance, after reviewing +that split, was not to ship it that way: a schema is either 2.x or pre-2.x, +DataJoint no longer supports pre-2.x, and landing the backward-compatible +subset alone is actively harmful here -- it clears the import error while +leaving any `longblob`/`attach` attributes in place, which silently corrupts +data instead of loudly failing to import. See +[the migration guide](https://docs.datajoint.com/how-to/migrate-to-v20/) for +the authoritative type mapping and phase structure this follows (this is +Phase I: code only, against empty schemas, no production data touched). + +Both `compat-fixes` and `datajoint-2.x` now point at the same commit and carry +the same content, kept as two names only so nothing that already referenced +either one breaks. + +### Full scope of this branch + +| change | count | +|---|---| +| `dj.schema` -> `dj.Schema` | 2 | +| `longblob` -> `` | 3 | +| `smallint` -> `int16` | 2 | +| `float` -> `float32` | 9 | +| `requirements.txt`: `datajoint>=0.13` -> `datajoint>=2.3` | — | ### The `longblob` change is not cosmetic Under DataJoint 2.x a `longblob` attribute is a **raw native column**. It -declares without error and the insert succeeds, but a numpy array written to it -comes back as `bytes`: +declares without error and the insert succeeds, but a numpy array written to +it comes back as `bytes`: ``` -longblob (as this element declares it) -> returned type: bytes round-trip OK: False +longblob (as this element declared it) -> returned type: bytes round-trip OK: False (the 2.x codec) -> returned type: ndarray round-trip OK: True ``` -Nothing raises. There is no warning. Existing pipeline code fails later, far -from the cause, or silently computes on the wrong thing. See the corresponding -issue on the upstream repo for the full round-trip evidence. +Nothing raises. There is no warning beyond a generic "consider a core +DataJoint type" notice at declaration time that says nothing about data loss. +Traced upstream in datajoint/datajoint-python#1527: PyMySQL has no encoder for +`np.ndarray` and silently falls back to `str(value)`; the same declaration on +PostgreSQL raises instead of corrupting. See +[datajoint/element-event#48](https://github.com/datajoint/element-event/issues/48) +for the full round-trip evidence. ### Using it @@ -145,6 +164,9 @@ pip install git+https://github.com/akshay-jaggi/element-event.git@datajoint-2.x ``` Pin the commit rather than the branch name if you need reproducibility. Every -change on both branches was applied mechanically with regexes and reviewed; no -element behaviour was altered, only attribute-type spellings that 2.x renamed or -replaced. +change was applied mechanically (regex over each table's `definition` string, +scoped so it cannot touch a docstring or a function signature) and then +reviewed line by line; no element behaviour was altered, only attribute-type +spellings that 2.x renamed, replaced, or requires as core types. + +Open PR: [datajoint/element-event#47](https://github.com/datajoint/element-event/pull/47).