Implement drivers for microwindows

[Acfboy]

Hello the community! Here is the work I have done in the early stage of GSoC.

Work Completed So Far

The GDI‑level Microwindows core, drivers, and mwdemo are functional on QEMU x86‑64 and the simulator.

Build System Integration

• Based on the initial proof-of-concept branch by @ppisa , the Microwindows core (engine, drivers, precompiled bitmap fonts) has been successfully compiled into the NuttX apps library.
• Resolved a build conflict: the ERROR macro in wingdi.h clashes with NuttX’s enum { ERROR = -1 }. The fix was to compile with -DNOGDI, a solution accepted by ghaerr and implemented upstream.
• Image resources used by mwdemo.c are compiled into binary using the convbmp tool during the build.

NuttX Hardware Drivers

Framebuffer driver (scr_nuttx.c), Keyboard driver (kbd_nuttx.c), Mouse driver (mou_nuttx.c), have been implemented to connect Microwindows to NuttX’s input/output subsystems.

Demo Application

Ported mwdemo.c (a complex Windows‑like demo) into a standalone NuttX example application. The demo runs successfully on both qemu‑intel64:mw and the NuttX simulator (sim:mw).

I have also tested the demo with #define CONTROL 1 enabled in mwdemo.c, which represents a more complex use case.

Upstream Bug Fixes

Several issues were discovered, analysed, and fixed – most of them already merged upstream:

Issue	Root Cause	Fix
Animation freezing in mwdemo	MwSelect() blocked forever when a timer had already expired, because the code assumed no timer existed when GdGetNextTimeout returned false.	Patch by ppisa (committed by ghaerr in PR #191): correctly handle the case where timeout == 0 and return immediately.
Cursor leaving a blue ghost image	#if defined(ELKS) in mwtypes.h forced MWPIXELVALHW to unsigned char even when ELKS=0 (intended to disable it). This truncated 32‑bit pixel values read from the framebuffer.	Changed to #if ELKS in ghaerr/microwindows@7a2ff57.
System freeze when dragging a window caption	During drag, MwSelect switches to polling (select(…, timeout=0)). The USB input thread had a lower priority (50) than the application (100), so it never got CPU time to generate events.	Short‑term: adjust thread priorities or add a minimal poll timeout. A permanent solution (refactoring the drag event loop) is under discussion. A NUTTX-specific patch will be prepared.

Remaining Issues

However, despite resolving several issues, two discovered problems are still under investigation. Due to a brief interruption while preparing for final exams, I have not yet fully diagnosed them. I will resume work soon and address the following:

1. When #define CONTROL 1 is enabled in mwdemo and input is tested in an edit box, backspace handling does not work correctly under QEMU.
2. In the simulator (sim), after dragging a window, the mouse cursor jumps outside the screen and drags the window off-screen as well.

Both issues appear to be related to the input drivers of the respective platforms. For instance, adding sample.point[0].x = x; sample.point[0].y = y; before the code at sim_touchscreen.c#L198-L220 resolves the mouse drift problem. It seems that sample.point[0] was previously uninitialized; however, the code branch in question never sets TOUCH_POS_VALID, so that value should not have been read in the first place. The root cause may lie elsewhere.

How to Reproduce (QEMU / Simulator)

qemu-intel64:mw

1. Download defconfig
2. Build nuttx

make distclean 
./tools/configure.sh qemu-intel64:mw
make -j$(nproc)

3. Creating a bootable disk following the instructions in the documentation
4. Run QEMU

qemu-system-x86_64 -cpu host -enable-kvm -m 2G \
        -cdrom boot_mw.iso \
         -serial stdio -monitor none -no-reboot -device qemu-xhci -device usb-kbd -device usb-mouse

sim:mw

1. Download defconfig
2. Build nuttx and run

make distclean 
./tools/configure.sh sim:mw
make -j$(nproc)
./nuttx

Notice

You can set #define CONTROL 1 manually in mwdemo.c to test more complex case.

Plan for the Second Half of GSoC

• Extend testing – verify on real hardware if possible; currently the work has been limited to QEMU and sim.
• Nano‑X (X11 API) port – enable Unix domain socket communication between the Nano‑X server and clients. The goal is to run multi‑client demos (e.g., src/demos/nanox/ applications).
• Driver submission to Microwindows mainline – after review by ghaerr, the NuttX screen/mouse/keyboard drivers should be integrated into the official Microwindows repository.
• Pull request to NuttX apps – submit a polished draft PR with all the work done so far, incorporating mentor feedback.
• Try to port FLTK gui framework and real-world applications.

[ppisa]

Thanks to @Acfboy for the work and this draft request for discussion and @ghaerr for consultations and support on Microwidows side.

I am leaving for four days now, but there are some my thoughts, the NuttX Microwidows screen, mouse and keyboard drivers should be submitted with appropriate build enable options to the mainline Microwindows https://github.com/ghaerr/microwindows before the final nuttx-apps pull request.

[acassis]

@Acfboy hello -> microwindows

[acassis]

@Acfboy please fix the issues on mwdemo:

[simbit18]

Hi @Acfboy , could you also add support for building with CMake?

[Acfboy]

Hi @Acfboy , could you also add support for building with CMake?

Thank you for the reminder. I will add CMake support later.

[acassis]

@Acfboy could you spend some time fixing and improving this PR? It still as Draft. It is important to get it merged and let more people to test and review it. Also please remember to include a proper nuttx/Documentation about nanox/microwindows support on NuttX

[Acfboy]

Hi @acassis , thank for the reminder. I agree I should submit the previous work as a proper PR for the community to review. However, for it to be a proper PR, I think I need to merge the existing NuttX drivers into the Microwindows mainline first. I apologize for postponing this part of the work earlier. I will submit a PR for upstream MW review soon.

[ppisa]

Yes, this what I have proposed and expect as well.

You should set something like SCREEN=NUTTX or SCREEN=NUTTX_FB in the local graphics/microwindows/Makefile and then Microwindows with merged code should add correct driver sources when you include microwindows/src/drivers/Objects.rules.

Same for the MOUSE and KEYBOARD. The question is if the mouse should be a single driver or one for relative motion mouse device and other for touchscreen.

As for the keyboard, there can be space to think how to map it the best way. The NnuttX provides its KEYCODE_xxx macros in kbd_codec.h for special keys. But only for these and rest is expected to be ASCII. Microwindows works with MWKEY and MWSCANCODE which aligns with concept of real X11 where the keyboards provide "position" of pressed key by scancode and it can be remapped to symbol even latter in application. But usually MWKEY code is used and its definitions are provided in src/include/mwtypes.h.

You provide translation in translate_keycode when keyboard is in g_kbd_mode set to KBD_MODE_EVENT but there is heuristics during open. I think that there could be two drivers, one for event type and another for raw keyboards. This could be configured through Kconfig, same would be useful for device name. So it is necessary to think a little what is the best option there.

You should try even if Nano-X server and clients can be run on NuttX. I do not see X11 graphics option as the main priority, but it would prove that even complex setup can be build and check if it is stable on NuttX.

[Acfboy]

The NnuttX provides its KEYCODE_xxx macros in kbd_codec.h for special keys. But only for these and rest is expected to be ASCII.

Hi, @ppisa . I have a question about this statement. It seems to me that the ranges of the KEYCODE_xxx macros and ASCII are conflicting, which makes it impossible for my driver to distinguish whether it's a character or a special key. In my sim:mw setup, this results in being unable to type a space.

I think that there could be two drivers, one for event type and another for raw keyboards. This could be configured through Kconfig, same would be useful for device name.

Two drivers makes the code cleaner. But specify the device path/name via Kconfig risks inconsistency with the real device. Alternatively, if Microwindows auto-selects the device based on NuttX config, then we have to keep both sides in sync — which adds maintenance overhead.

You should try even if Nano-X server and clients can be run on NuttX.

Okay, I'll try server and client of NanoX.

[ppisa]

The NnuttX provides its KEYCODE_xxx macros in kbd_codec.h for special keys. But only for these and rest is expected to be ASCII.

Hi, @ppisa . I have a question about this statement. It seems to me that the ranges of the KEYCODE_xxx macros and ASCII are conflicting, which makes it impossible for my driver to distinguish whether it's a character or a special key. In my sim:mw setup, this results in being unable to type a space.

Yes, you are right, I expected that for KEYCODE_NORMAL the ASCII value is returned in another field of the event structure, but there is only

struct keyboard_event_s
{
  uint32_t type;
  uint32_t code;
};

where type is KEYBOARD_PRESS or KEYBOARD_RELEASE. But into code sent to keyboard_event() is ASCII code for example in g_km_keymap in STM32 matrix keyboard but some other drivers call keyboard_event() even with X11 encoded KeySim (arch/sim/src/sim/posix/sim_x11eventloop.c) and some with keys translated to KEYCODE_xxx where is no space for ASCII.

So this seems to be call for priority issue.

@acassis @gregory-nutt Please, do you have some some insight what is right?

The simple fix is to push KEYCODE_FWDDEL in include/nuttx/input/kbd_codec.h to be above ASCII, for example 128 or 129. But may it be it can break some targets where it would result in some large arrays, etc. When keyboards with interpreted national symbols are used then it would worth to push special codes even further after basic unicode...

I think that there could be two drivers, one for event type and another for raw keyboards. This could be configured through Kconfig, same would be useful for device name.

Two drivers makes the code cleaner. But specify the device path/name via Kconfig risks inconsistency with the real device. Alternatively, if Microwindows auto-selects the device based on NuttX config, then we have to keep both sides in sync — which adds maintenance overhead.

Yes but NuttX is based on configuration and it is better to fail then to do lot of testing of different device names and even deciding if it is raw driver or event based. But in general it seems that there is pace for some discussion and making KBD simple on the NuttX side.

You should try even if Nano-X server and clients can be run on NuttX.

Okay, I'll try server and client of NanoX.

Thanks.