Cookbook of device patterns, lot 2: vendor DLL (ctypes) and multi-axis controller

README.rst

@@ -178,9 +178,10 @@ Cookbook of device patterns
 
 Short, runnable and tested plugins to copy from when writing a plugin for a real instrument. Each recipe is a pair
 (actuator and 0D detector) that runs without hardware: leave the port or the address to ``MOCK`` and the plugin talks
-to a simulated stage with a voltmeter (``hardware/mock_serial_visa_instrument.py``). Each file starts with a header telling what to
-copy, what to adapt and what to keep. The tests (``tests/test_cookbook_drivers.py``, ``tests/test_cookbook_plugins.py``)
-show how to test a plugin against a simulated device with the harness of ``pymodaq.utils.plugin_testing``.
+to a simulated device: a stage with a voltmeter (``hardware/mock_serial_visa_instrument.py``) or a 3-axis controller
+behind a library (``hardware/mock_dll_controller.py``). Each file starts with a header telling what to
+copy, what to adapt and what to keep. The tests (``tests/test_cookbook_drivers.py``, ``tests/test_cookbook_plugins.py``,
+``tests/test_cookbook_dll.py``) show how to test a plugin against a simulated device with the harness of ``pymodaq.utils.plugin_testing``.
 
 Giving the same *controller ID* to an actuator and a detector of a recipe makes them share one driver: the first
 initialized is the master and creates it, the other is the slave and receives it.
@@ -198,6 +199,16 @@ driver written directly                 ``daq_0Dviewer_CookbookSerial``,
 VISA / SCPI, driver written directly    ``daq_move_CookbookVisa``,                      The same on a ``pyvisa`` resource: explicit terminations and
                                         ``daq_0Dviewer_CookbookVisa``,                  timeout, ``query`` versus ``write``
                                         ``hardware/visa_meter.py``
+Vendor DLL, multi-axis controller,      ``daq_move_CookbookDll`` (3 axes),              A ``ctypes`` driver: ``argtypes``, ``restype``, opaque handle,
+``ctypes``                              ``daq_0Dviewer_CookbookDll``,                   ``byref``, a structure, error codes, a ``CFUNCTYPE`` callback
+                                        ``hardware/dll_controller.py``                  kept alive. The library is simulated by an object that behaves
+                                                                                        like a ``CDLL``. The end of a move is reported by a callback
+                                                                                        from a thread of the library, turned into a Qt signal. Axes
+                                                                                        share one handle (master and slaves)
+Loading a native library                ``hardware/native_library.py``,                 ``load_library``: names per platform, path from a setting or an
+                                        ``hardware/system_library_demo.py``             environment variable, an error that lists what was tried. Real
+                                                                                        calls to the C library of the system (``strlen``, ``frexp``,
+                                                                                        ``qsort`` with a callback)
 ======================================  ==============================================  ================================================================
 
 Patterns already shown by the mock instruments of this package
@@ -219,7 +230,8 @@ Simulated device, no hardware           every plugin of this package
                                                                                         ``hardware/``: the plugins run on it, no hardware needed
 ======================================  ==============================================  ================================================================
 
-Patterns not yet covered: an instrument driven by a vendor DLL or SDK (``ctypes`` or a Python binding).
+Patterns not yet covered: an instrument driven by a Python binding of the manufacturer (it is packaged like the
+drivers of ``hardware/``: a class that wraps the binding, created by ``ini_stage``).
 
 
 PID Models

src/pymodaq_plugins_mockexamples/daq_move_plugins/daq_move_CookbookDll.py

@@ -0,0 +1,116 @@
+"""Cookbook recipe 2a: multi-axis actuator on a vendor DLL (``ctypes``), with a move that ends by a callback.
+
+COPY this file for an actuator whose manufacturer supplies a C library (DLL, shared object) for a controller with
+several axes.
+ADAPT: ``params`` (what the library needs to connect), ``_axis_names`` (axis name -> the number the library uses),
+``_controller_units``, the calls of the driver in ``move_abs``, ``move_rel``, ``get_actuator_value``, ``move_home`` and
+``stop_motion``.
+KEEP:
+- the master/slave handling in ``ini_stage`` and ``close``: the first axis (master) opens the controller, the others
+  (slaves, same controller ID in the Dashboard) receive it;
+- the callback handling. The library calls back from one of its own threads: ``_forward_event`` only emits a Qt signal,
+  which is delivered to ``_on_motion_event`` in the thread of the plugin. Never call ``move_done`` or touch a widget
+  from the library thread;
+- ``user_condition_to_reach_target``: PyMoDAQ polls the position until it is within epsilon of the target, and also
+  waits for the end-of-move event. The homing is not polled, so ``move_done`` is called when its event arrives.
+
+Try it without the DLL: leave the port to ``MOCK``, the library is then the simulated one of
+``hardware/mock_dll_controller.py``. See ``hardware/dll_controller.py`` for the ``ctypes`` driver and
+``hardware/native_library.py`` for the loading of the library.
+"""
+from qtpy.QtCore import Signal, Slot
+
+from pymodaq.control_modules.move_utility_classes import (DAQ_Move_base, comon_parameters_fun, DataActuatorType,
+                                                          DataActuator, main)
+
+from pymodaq_plugins_mockexamples.hardware.dll_controller import MotionController, open_controller
+from pymodaq_plugins_mockexamples.hardware.mock_dll_controller import (EVENT_MOVE_DONE, EVENT_STOPPED,
+                                                                       EVENT_HOME_DONE)
+
+
+class DAQ_Move_CookbookDll(DAQ_Move_base):
+    """One axis of the simulated 3-axis controller, driven through ``MotionController`` on a DLL"""
+    _controller_units = 'mm'
+    is_multiaxes = True
+    _axis_names = {'X': 0, 'Y': 1, 'Z': 2}  # the library numbers its axes
+    _epsilons = 0.01
+    data_actuator_type = DataActuatorType.DataActuator
+    params = comon_parameters_fun(is_multiaxes, axis_names=_axis_names, epsilon=_epsilons) + [
+        {'title': 'Port:', 'name': 'port', 'type': 'str', 'value': 'MOCK',
+         'tip': 'Port of the controller, MOCK for the simulated library'},
+        {'title': 'Library path:', 'name': 'library_path', 'type': 'browsepath', 'value': '', 'filetype': True,
+         'tip': 'Full path to the DLL, empty to search it (environment variable MM_LIBRARY, system folders)'},
+    ]
+
+    motion_event = Signal(int, int)  # axis, event: emitted from the thread of the library, received in this thread
+
+    def ini_attributes(self):
+        self.controller: MotionController = None
+        self._move_finished = True
+        self._homing = False
+        self.motion_event.connect(self._on_motion_event)
+
+    def ini_stage(self, controller=None):
+        if self.is_master:
+            self.controller = open_controller(self.settings['port'], self.settings['library_path'])
+        else:  # a master axis shares its controller
+            self.controller = controller
+        self.controller.subscribe(self._forward_event)
+        info = self.controller.info()
+        return f'{info.name}, firmware {info.firmware[0]}.{info.firmware[1]}', True
+
+    def close(self):
+        if self.controller is not None:
+            self.controller.unsubscribe(self._forward_event)
+            if self.is_master:
+                self.controller.close()
+
+    def _forward_event(self, axis: int, event: int):
+        """Called from the thread of the library: only emit the signal"""
+        self.motion_event.emit(axis, event)
+
+    @Slot(int, int)
+    def _on_motion_event(self, axis: int, event: int):
+        """Runs in the thread of the plugin"""
+        if axis != self.axis_value:  # the controller is shared: events of the other axes come too
+            return
+        if event == EVENT_HOME_DONE and self._homing:
+            self._homing = False
+            self.move_done()
+        elif event in (EVENT_MOVE_DONE, EVENT_STOPPED):
+            self._move_finished = True
+
+    def user_condition_to_reach_target(self) -> bool:
+        return self._move_finished
+
+    def get_actuator_value(self) -> DataActuator:
+        position = DataActuator(data=self.controller.position(self.axis_value), units=self.axis_unit)
+        return self.get_position_with_scaling(position)
+
+    def move_abs(self, value: DataActuator):
+        value = self.check_bound(value)
+        self.target_value = value
+        value = self.set_position_with_scaling(value)
+        self._move_finished = False
+        self.controller.move_abs(self.axis_value, value.value(self.axis_unit))
+
+    def move_rel(self, value: DataActuator):
+        value = self.check_bound(self.current_value + value) - self.current_value
+        self.target_value = value + self.current_value
+        value = self.set_position_with_scaling(self.target_value)
+        self._move_finished = False
+        self.controller.move_abs(self.axis_value, value.value(self.axis_unit))
+
+    def move_home(self):
+        self._homing = True
+        self.controller.home(self.axis_value)
+
+    def stop_motion(self):
+        self.controller.stop(self.axis_value)
+
+    def commit_settings(self, param):
+        pass
+
+
+if __name__ == '__main__':
+    main(__file__)

src/pymodaq_plugins_mockexamples/daq_viewer_plugins/plugins_0D/daq_0Dviewer_CookbookDll.py

@@ -0,0 +1,57 @@
+"""Cookbook recipe 2b: 0D detector reading the positions of the axes of a controller driven by a vendor DLL.
+
+COPY this file for a detector of a controller that has several functions (here the encoders of three axes) behind one
+library handle. Alone it opens its own controller (master); with the same controller ID as the actuators
+``CookbookDll`` it is a slave and shares their handle: a library handle must be opened once, not once per plugin.
+ADAPT: ``params``, ``ini_detector``, and the acquisition in ``grab_data`` (name, labels and units of the data).
+KEEP: the master/slave handling in ``ini_detector`` and ``close``, and the units given to the data.
+"""
+import numpy as np
+
+from pymodaq.control_modules.viewer_utility_classes import DAQ_Viewer_base, comon_parameters, main
+from pymodaq.utils.data import DataFromPlugins
+from pymodaq_data import DataToExport
+
+from pymodaq_plugins_mockexamples.hardware.dll_controller import MotionController, open_controller
+
+
+class DAQ_0DViewer_CookbookDll(DAQ_Viewer_base):
+    """Positions of the three axes of the simulated controller, read through ``MotionController``"""
+    params = comon_parameters + [
+        {'title': 'Port:', 'name': 'port', 'type': 'str', 'value': 'MOCK',
+         'tip': 'Port of the controller, MOCK for the simulated library'},
+        {'title': 'Library path:', 'name': 'library_path', 'type': 'browsepath', 'value': '', 'filetype': True,
+         'tip': 'Full path to the DLL, empty to search it (environment variable MM_LIBRARY, system folders)'},
+    ]
+
+    def ini_attributes(self):
+        self.controller: MotionController = None
+
+    def ini_detector(self, controller=None):
+        if self.is_master:
+            self.controller = open_controller(self.settings['port'], self.settings['library_path'])
+        else:  # a master actuator or detector shares its controller
+            self.controller = controller
+        info = self.controller.info()
+        return f'{info.name}, firmware {info.firmware[0]}.{info.firmware[1]}', True
+
+    def close(self):
+        if self.is_master and self.controller is not None:
+            self.controller.close()
+
+    def grab_data(self, Naverage=1, **kwargs):
+        positions = [np.array([self.controller.position(axis)]) for axis in range(self.controller.n_axes)]
+        self.dte_signal.emit(DataToExport(
+            name='CookbookDll',
+            data=[DataFromPlugins(name='Positions', data=positions, dim='Data0D', labels=['X', 'Y', 'Z'],
+                                  units='mm')]))
+
+    def stop(self):
+        pass
+
+    def commit_settings(self, param):
+        pass
+
+
+if __name__ == '__main__':
+    main(__file__)

src/pymodaq_plugins_mockexamples/hardware/dll_controller.py

@@ -0,0 +1,130 @@
+"""Driver of the 3-axis motion controller of ``mock_dll_controller.py``, written like a driver for a vendor DLL.
+
+This is the ``ctypes`` layer that a plugin author writes (or finds written) on a manufacturer library:
+
+- ``_declare`` gives every function its ``argtypes`` and ``restype``: without them ``ctypes`` guesses (C ``int``
+  arguments and result) and a 64 bits pointer or a ``double`` is silently corrupted;
+- the library hands out an opaque *handle* (``void *``) received through ``byref`` and given back to every call;
+- every function returns an integer error code, turned into an exception by ``_check`` with the message of the library;
+- the library fills a structure (``MM_Info``);
+- a callback made with ``CFUNCTYPE`` is registered once and kept in an attribute: if Python collects it while the
+  library still holds the pointer, the next call crashes the interpreter. It is called from a thread of the library,
+  so it only forwards the event to the listeners, which must not touch Qt widgets.
+
+Load the real library with ``MotionController.from_library(load_library('mm.dll'))`` (see ``native_library.py``) or
+the simulated one with ``MotionController.mock()``.
+"""
+import ctypes
+from dataclasses import dataclass
+from typing import Callable
+
+from pymodaq_plugins_mockexamples.hardware.mock_dll_controller import (MM_CALLBACK, MM_Info, MockMotionLibrary, MM_OK,
+                                                                       PORT, N_AXES)
+
+
+class MotionControllerError(RuntimeError):
+    """An error code returned by the library, with the message of the library"""
+
+
+@dataclass
+class ControllerInfo:
+    name: str
+    n_axes: int
+    firmware: tuple[int, int]
+
+
+class MotionController:
+    """Handle of an opened controller and its axes (numbered from 0)"""
+    n_axes = N_AXES
+
+    def __init__(self, library, port: str = PORT):
+        self._lib = library
+        self._declare()
+        self._handle = ctypes.c_void_p()
+        self._check(self._lib.mm_open(port.encode(), ctypes.byref(self._handle)))
+        self._listeners: list[Callable[[int, int], None]] = []
+        self._callback = MM_CALLBACK(self._on_event)  # kept alive: the library keeps the pointer
+        self._check(self._lib.mm_set_callback(self._handle, self._callback))
+
+    @classmethod
+    def mock(cls, speed: float = 25.) -> 'MotionController':
+        """The controller on the simulated library (no DLL needed)"""
+        return cls(MockMotionLibrary(speed))
+
+    def _declare(self):
+        lib = self._lib
+        lib.mm_open.argtypes = [ctypes.c_char_p, ctypes.POINTER(ctypes.c_void_p)]
+        lib.mm_close.argtypes = [ctypes.c_void_p]
+        lib.mm_get_info.argtypes = [ctypes.c_void_p, ctypes.POINTER(MM_Info)]
+        lib.mm_move_abs.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_double]
+        lib.mm_get_position.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.POINTER(ctypes.c_double)]
+        lib.mm_is_moving.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.POINTER(ctypes.c_int)]
+        lib.mm_stop.argtypes = [ctypes.c_void_p, ctypes.c_int]
+        lib.mm_home.argtypes = [ctypes.c_void_p, ctypes.c_int]
+        lib.mm_set_callback.argtypes = [ctypes.c_void_p, MM_CALLBACK]
+        lib.mm_error_message.argtypes = [ctypes.c_int]
+        lib.mm_error_message.restype = ctypes.c_char_p
+        for name in ('mm_open', 'mm_close', 'mm_get_info', 'mm_move_abs', 'mm_get_position', 'mm_is_moving', 'mm_stop',
+                     'mm_home', 'mm_set_callback'):
+            getattr(lib, name).restype = ctypes.c_int
+
+    def _check(self, code: int):
+        if code != MM_OK:
+            raise MotionControllerError(f'{self._lib.mm_error_message(code).decode()} (code {code})')
+
+    def _on_event(self, axis: int, event: int):
+        """Called by the library, from one of its threads"""
+        for listener in list(self._listeners):
+            listener(axis, event)
+
+    def subscribe(self, listener: Callable[[int, int], None]):
+        """``listener(axis, event)`` is called, from a thread of the library, when a move ends (see the EVENT_*)"""
+        self._listeners.append(listener)
+
+    def unsubscribe(self, listener: Callable[[int, int], None]):
+        if listener in self._listeners:
+            self._listeners.remove(listener)
+
+    def info(self) -> ControllerInfo:
+        info = MM_Info()
+        self._check(self._lib.mm_get_info(self._handle, ctypes.byref(info)))
+        return ControllerInfo(info.name.decode(), info.n_axes, (info.firmware[0], info.firmware[1]))
+
+    def position(self, axis: int) -> float:
+        position = ctypes.c_double()
+        self._check(self._lib.mm_get_position(self._handle, axis, ctypes.byref(position)))
+        return position.value
+
+    def is_moving(self, axis: int) -> bool:
+        moving = ctypes.c_int()
+        self._check(self._lib.mm_is_moving(self._handle, axis, ctypes.byref(moving)))
+        return bool(moving.value)
+
+    def move_abs(self, axis: int, position: float):
+        """Start the move; the library calls back when it is over"""
+        self._check(self._lib.mm_move_abs(self._handle, axis, position))
+
+    def stop(self, axis: int):
+        self._check(self._lib.mm_stop(self._handle, axis))
+
+    def home(self, axis: int):
+        """Start the homing; the library calls back when it is over"""
+        self._check(self._lib.mm_home(self._handle, axis))
+
+    def close(self):
+        if self._handle.value is not None:
+            self._lib.mm_close(self._handle)
+            self._handle = ctypes.c_void_p()
+        self._listeners.clear()
+
+
+LIBRARY_NAMES = {'Windows': ['mm.dll'], 'Linux': ['libmm.so'], 'Darwin': ['libmm.dylib']}  # names of the (imaginary) DLL
+
+
+def open_controller(port: str, library_path: str = '') -> MotionController:
+    """The controller of the simulated library for the port ``MOCK``, else of the vendor library found by
+    ``load_library`` (setting of the plugin, environment variable ``MM_LIBRARY``, system folders)"""
+    if port == PORT:
+        return MotionController.mock()
+    from pymodaq_plugins_mockexamples.hardware.native_library import load_library
+    return MotionController(load_library(LIBRARY_NAMES, path=library_path or None, env_var='MM_LIBRARY'), port)