Hello everyone,
I was trying to load a recording object from an NWB file and got an error. For context, this electrical series is the result of using spikeinterface to process (filtering & motion correction) and sort the units. At the end of the sorting process, we export the units and the processed recording back into the original NWB (following NEUROCONV naming convention). As a result of the processing with spikeinterface, the channels were sorted by their location on the probe. However, the electrode table in the NWB preserves the recording system ordering. When trying to load the electrical series with spikeinterface we got an error that electrode_indices are not in increasing order, which makes sense because they were sorted by location.
To avoid this restriction, I added "[:]" to extract the data into a numpy array, and then the indices do not need to be in increasing order. I'm not sure if this will cause issues while loading other properties, but it works for us for now. But if this is not the proper way to do it, we will wait for any further comments.
I made the changes in the following lines:
spikeinterface.version = 0.102.3
spikeinterface\extractors\nwbextractors.py
Line 774: channel_names = electrodes_table["channel_name"]
new: channel_names = electrodes_table["channel_name"][:]
Line 798: locations[:, 0] = electrodes_table["rel_x"][electrodes_indices]
new: locations[:, 0] = electrodes_table["rel_x"][:][electrodes_indices]
Line 799: locations[:, 1] = electrodes_table["rel_y"][electrodes_indices]
new: locations[:, 1] = electrodes_table["rel_y"][:][electrodes_indices]
Line 801: locations[:, 2] = electrodes_table["rel_z"][electrodes_indices]
new: locations[:, 2] = electrodes_table["rel_z"][:][electrodes_indices]
Line 806: groups = electrodes_table["group_name"][electrodes_indices][:]
new: groups = electrodes_table["group_name"][:][electrodes_indices][:]
Line 833: properties[column_name] = electrodes_table[column][electrodes_indices]
new: properties[column_name] = electrodes_table[column][:][electrodes_indices]
Here is the error that I got when using "read_nwb":
----> 1 si_recording = read_nwb(
2 file_path=NWBfile_path,
3 electrical_series_path='processing/ecephys/Processed/' + 'ephys-{}'.format(electrodeGroupName),
4 load_recording=True,
5 load_sorting=False
6 )
8 si_recording
File m:\YauLab_Python\SIenv\Lib\site-packages\spikeinterface\extractors\nwbextractors.py:1771, in read_nwb(file_path, load_recording, load_sorting, electrical_series_path)
1769 outputs = ()
1770 if load_recording:
-> 1771 rec = read_nwb_recording(file_path, electrical_series_path=electrical_series_path)
1772 outputs = outputs + (rec,)
1773 if load_sorting:
File m:\YauLab_Python\SIenv\Lib\site-packages\spikeinterface\extractors\nwbextractors.py:590, in NwbRecordingExtractor.__init__(self, file_path, electrical_series_name, load_time_vector, samples_for_rate_estimation, stream_mode, stream_cache_path, electrical_series_path, load_channel_properties, file, cache, storage_options, use_pynwb)
576 (
577 channel_ids,
578 sampling_frequency,
(...)
581 times_kwargs,
582 ) = self._fetch_recording_segment_info_pynwb(file, cache, load_time_vector, samples_for_rate_estimation)
583 else:
584 (
585 channel_ids,
586 sampling_frequency,
587 dtype,
588 segment_data,
589 times_kwargs,
--> 590 ) = self._fetch_recording_segment_info_backend(file, cache, load_time_vector, samples_for_rate_estimation)
592 BaseRecording.__init__(self, channel_ids=channel_ids, sampling_frequency=sampling_frequency, dtype=dtype)
593 recording_segment = NwbRecordingSegment(
594 electrical_series_data=segment_data,
595 times_kwargs=times_kwargs,
596 )
File m:\YauLab_Python\SIenv\Lib\site-packages\spikeinterface\extractors\nwbextractors.py:775, in NwbRecordingExtractor._fetch_recording_segment_info_backend(self, file, cache, load_time_vector, samples_for_rate_estimation)
773 if "channel_name" in electrode_table_columns:
774 channel_names = electrodes_table["channel_name"]
--> 775 channel_ids = channel_names[electrodes_indices]
776 # Decode if bytes with utf-8
777 channel_ids = [x.decode("utf-8") if isinstance(x, bytes) else x for x in channel_ids]
File h5py\\_objects.pyx:54, in h5py._objects.with_phil.wrapper()
File h5py\\_objects.pyx:55, in h5py._objects.with_phil.wrapper()
File m:\YauLab_Python\SIenv\Lib\site-packages\h5py\_hl\dataset.py:831, in Dataset.__getitem__(self, args, new_dtype)
826 return arr
828 # === Everything else ===================
829
830 # Perform the dataspace selection.
--> 831 selection = sel.select(self.shape, args, dataset=self)
833 if selection.nselect == 0:
834 return numpy.zeros(selection.array_shape, dtype=new_dtype)
File m:\YauLab_Python\SIenv\Lib\site-packages\h5py\_hl\selections.py:82, in select(shape, args, dataset)
79 space = h5s.create_simple(shape)
80 selector = _selector.Selector(space)
---> 82 return selector.make_selection(args)
File h5py\\_selector.pyx:282, in h5py._selector.Selector.make_selection()
File h5py\\_selector.pyx:215, in h5py._selector.Selector.apply_args()
TypeError: Indexing elements must be in increasing orderAnd I got this one when I tried with "read_nwb_recording"
TypeError Traceback (most recent call last)
Cell In[5], line 1
----> 1 si_recording = read_nwb_recording(
2 file_path=NWBfile_path,
3 electrical_series_path='processing/ecephys/Processed/' + 'ephys-{}'.format(electrodeGroupName),
4 use_pynwb=True
5 )
7 si_recording
File m:\YauLab_Python\SIenv\Lib\site-packages\spikeinterface\extractors\nwbextractors.py:601, in NwbRecordingExtractor.__init__(self, file_path, electrical_series_name, load_time_vector, samples_for_rate_estimation, stream_mode, stream_cache_path, electrical_series_path, load_channel_properties, file, cache, storage_options, use_pynwb)
599 # fetch and add main recording properties
600 if use_pynwb:
--> 601 gains, offsets, locations, groups = self._fetch_main_properties_pynwb()
602 else:
603 gains, offsets, locations, groups = self._fetch_main_properties_backend()
File m:\YauLab_Python\SIenv\Lib\site-packages\spikeinterface\extractors\nwbextractors.py:860, in NwbRecordingExtractor._fetch_main_properties_pynwb(self)
857 offset = electrodes_table["offset"].data[electrodes_indices]
858 offsets = offset * 1e6
--> 860 locations, groups = self._fetch_locations_and_groups(electrodes_table, electrodes_indices)
862 return gains, offsets, locations, groups
File m:\YauLab_Python\SIenv\Lib\site-packages\spikeinterface\extractors\nwbextractors.py:798, in NwbRecordingExtractor._fetch_locations_and_groups(self, electrodes_table, electrodes_indices)
796 ndim = 3 if "rel_z" in electrodes_table else 2
797 locations = np.zeros((self.get_num_channels(), ndim), dtype=float)
--> 798 locations[:, 0] = electrodes_table["rel_x"][electrodes_indices]
799 locations[:, 1] = electrodes_table["rel_y"][electrodes_indices]
800 if "rel_z" in electrodes_table:
File m:\YauLab_Python\SIenv\Lib\site-packages\hdmf\container.py:974, in Data.__getitem__(self, args)
973 def __getitem__(self, args):
--> 974 return self.get(args)
File m:\YauLab_Python\SIenv\Lib\site-packages\hdmf\common\table.py:61, in VectorData.get(self, key, **kwargs)
54 def get(self, key, **kwargs):
55 """
56 Retrieve elements from this VectorData
57
58 :param key: Selection of the elements
59 :param kwargs: Ignored
60 """
---> 61 return super().get(key)
File m:\YauLab_Python\SIenv\Lib\site-packages\hdmf\container.py:982, in Data.get(self, args)
979 if isinstance(self.data, h5py.Dataset) and isinstance(args, np.ndarray):
980 # This is needed for h5py 2.9 compatibility
981 args = args.tolist()
--> 982 return self.data[args]
File h5py\\_objects.pyx:54, in h5py._objects.with_phil.wrapper()
File h5py\\_objects.pyx:55, in h5py._objects.with_phil.wrapper()
File m:\YauLab_Python\SIenv\Lib\site-packages\h5py\_hl\dataset.py:831, in Dataset.__getitem__(self, args, new_dtype)
826 return arr
828 # === Everything else ===================
829
830 # Perform the dataspace selection.
--> 831 selection = sel.select(self.shape, args, dataset=self)
833 if selection.nselect == 0:
834 return numpy.zeros(selection.array_shape, dtype=new_dtype)
File m:\YauLab_Python\SIenv\Lib\site-packages\h5py\_hl\selections.py:82, in select(shape, args, dataset)
79 space = h5s.create_simple(shape)
80 selector = _selector.Selector(space)
---> 82 return selector.make_selection(args)
File h5py\\_selector.pyx:282, in h5py._selector.Selector.make_selection()
File h5py\\_selector.pyx:215, in h5py._selector.Selector.apply_args()
TypeError: Indexing elements must be in increasing orderThanks in advance for any suggestions.
Best
Pepe
Hello everyone,
I was trying to load a recording object from an NWB file and got an error. For context, this electrical series is the result of using spikeinterface to process (filtering & motion correction) and sort the units. At the end of the sorting process, we export the units and the processed recording back into the original NWB (following NEUROCONV naming convention). As a result of the processing with spikeinterface, the channels were sorted by their location on the probe. However, the electrode table in the NWB preserves the recording system ordering. When trying to load the electrical series with spikeinterface we got an error that electrode_indices are not in increasing order, which makes sense because they were sorted by location.
To avoid this restriction, I added "[:]" to extract the data into a numpy array, and then the indices do not need to be in increasing order. I'm not sure if this will cause issues while loading other properties, but it works for us for now. But if this is not the proper way to do it, we will wait for any further comments.
I made the changes in the following lines:
spikeinterface.version = 0.102.3
spikeinterface\extractors\nwbextractors.py
Here is the error that I got when using "read_nwb":
And I got this one when I tried with "read_nwb_recording"
Thanks in advance for any suggestions.
Best
Pepe