[DRAFT/TESTING] userspace LL/audio test PR (latest ver: V38) - #10558
[DRAFT/TESTING] userspace LL/audio test PR (latest ver: V38)#10558kv2019i wants to merge 10 commits into
Conversation
|
Running the pipeline_two_components_user test added in this PR, on a Intel PTL, looks like this: |
9cdb1be to
37b0412
Compare
|
V2 snapshot pushed:
|
37b0412 to
7317b18
Compare
|
V3 snapshot pushed:
|
|
Test output for the new test added in V3: |
| /* works? yes */ | ||
| //return 0; | ||
|
|
||
| printk("ipc %p\n", ipc); |
There was a problem hiding this comment.
Oops, these shouldn't be here. :)
|
|
||
| /* create the pipeline */ | ||
| pipe = pipeline_new(NULL, pipe_desc->primary.r.instance_id, | ||
| pipe = pipeline_new(heap, pipe_desc->primary.r.instance_id, |
There was a problem hiding this comment.
@lyakh @jsarha @lgirdwood This is where you'd plug in the vregions stuff to pass a separate heap to each pipe (based on topology description of its needs). In this series, as a placeholder, I use the zephyr_ll_user_heap() instead.
There was a problem hiding this comment.
right, maybe put a comment there for now to make re-discovery easier
lyakh
left a comment
There was a problem hiding this comment.
last reviewed commit so far "schedule: zephyr_ll: implement thread_init/free domain ops"
| #ifdef CONFIG_SOF_USERSPACE_LL | ||
| void comp_grant_access_to_thread(const struct comp_dev *dev, struct k_thread *th) | ||
| { | ||
| assert(dev->list_mutex); |
There was a problem hiding this comment.
description a bit confusing - this is only granting access to a mutex. Also list_mutex is only added to comp_dev in the next commit.
| } | ||
|
|
||
| stream_addr = rballoc_align(flags, size, align); | ||
| stream_addr = sof_heap_alloc(heap, flags, size, align); |
There was a problem hiding this comment.
the commit, that is mentioned in the commit message, only moved buffer context objects to particular heaps. This commit moves actual data buffers to them too, which is different and (arguably) more risky
| #define HDA_DMA_BUFFER_PERIOD_COUNT 4 | ||
|
|
||
| SHARED_DATA struct sof_dma dma[] = { | ||
| APP_TASK_DATA SHARED_DATA struct sof_dma dma[] = { |
There was a problem hiding this comment.
do I understand correctly, that this kind of userspace access makes that data writable to userspace?
| comp_err(dev, "requested channel %d is busy", hda_chan); | ||
| return -ENODEV; | ||
| } | ||
| hd->chan = &hd->dma->chan[channel]; |
There was a problem hiding this comment.
is this also not needed for the legacy mode? Also below
| uint64_t next_sync; | ||
| uint64_t period_in_cycles; | ||
| #endif | ||
| struct k_heap *heap; |
There was a problem hiding this comment.
wasn't this already referenced in the previous commit?
|
|
||
| k_spinlock_init(&dd->dai->lock); | ||
| #ifdef CONFIG_SOF_USERSPACE_LL | ||
| dd->dai->lock = k_object_alloc(K_OBJ_MUTEX); |
There was a problem hiding this comment.
check for NULL? Possibly in other locations too
| k_mutex_lock(dai->lock, K_FOREVER); | ||
| props = dai_get_properties(dai->dev, direction, stream_id); | ||
| hs_id = props->dma_hs_id; | ||
| ret = dai_get_properties_copy(dai->dev, direction, stream_id, &props); |
There was a problem hiding this comment.
I'm guessing this is made a syscall in one of the commits
| mod_heap = &mod_heap_user->heap; | ||
| } else { | ||
| #ifdef CONFIG_SOF_USERSPACE_LL | ||
| mod_heap = zephyr_ll_user_heap(); |
There was a problem hiding this comment.
looks good, but this else is entered under multiple conditions, might need to double-check
| .schedule_task_before = zephyr_ll_task_schedule_before, | ||
| .schedule_task_after = zephyr_ll_task_schedule_after, | ||
| .schedule_task_free = zephyr_ll_task_free, | ||
| .schedule_task_free = zephyr_ll_task_sched_free, |
There was a problem hiding this comment.
let's "spend" 3 more characters and make it ..._schedule_free()
| return -ENOMEM; | ||
| } | ||
| tr_err(&ll_tr, "Failed to allocate thread object for core %d", core); | ||
| dt->handler = NULL; |
| { | ||
| const struct sof_man_fw_desc *desc = basefw_vendor_get_manifest(); | ||
| const struct sof_man_module *mod; | ||
| uint32_t i; |
| uint32_t i; | ||
|
|
||
| if (!desc) | ||
| return -1; |
| return (int)i; | ||
| } | ||
|
|
||
| return -1; |
| union ipc4_connector_node_id node_id; | ||
| uint32_t dma_buffer_size; | ||
| uint32_t config_length; | ||
| } __packed __aligned(4); |
There was a problem hiding this comment.
does __aligned actually make sense in a type definition?
| pipe_msg.extension.dat = ipc_user->ipc_msg_ext; | ||
|
|
||
| /* Execute pipeline creation in user context */ | ||
| ipc_user->result = ipc_pipeline_new(ipc_user->ipc, (ipc_pipe_new *)&pipe_msg); |
There was a problem hiding this comment.
that's brave! ;-) I'd put a huge "TODO" here to make sure not to ship this by chance :-)
|
|
||
| /* create the pipeline */ | ||
| pipe = pipeline_new(NULL, pipe_desc->primary.r.instance_id, | ||
| pipe = pipeline_new(heap, pipe_desc->primary.r.instance_id, |
There was a problem hiding this comment.
right, maybe put a comment there for now to make re-discovery easier
7317b18 to
c029d49
Compare
|
V4 snapshot pushed:
|
|
Example output with V4 patchset: |
c029d49 to
9c778e4
Compare
|
V5 pushed:
|
9c778e4 to
c4b127f
Compare
|
V6 pushed:
|
c4b127f to
ad359d6
Compare
|
V7 submitted:
|
ad359d6 to
c7e9a37
Compare
|
V8 submitted:
|
c7e9a37 to
8af1be4
Compare
|
V9 submitted:
|
|
V10:
|
aa5efe0 to
35a61b6
Compare
|
V11 pushed:
|
35a61b6 to
214c051
Compare
|
V12 pushed:
|
Place the pipeline position lookup table in the sysuser memory partition and replace k_spinlock with a dynamically allocated k_mutex when CONFIG_SOF_USERSPACE_LL is enabled. Spinlocks disable interrupts which is a privileged operation unavailable from user-mode threads. The mutex pointer is stored in a separate APP_SYSUSER_BSS variable outside the SHARED_DATA struct so Zephyr's kernel object tracking can recognize it for syscall verification. Move pipeline_posn_init() from task_main_start() to primary_core_init() before platform_init(), so the mutex is allocated before ipc_user_init() grants thread access to it. In pipeline_posn_get(), bypass the sof_get() kernel singleton and access the shared structure directly when running in user-space. Grant the ipc_user_init thread access to the pipeline position mutex via new pipeline_posn_grant_access() helper. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add a check to ensure task state is what is expected after k_sem_take() returns in zephyr_ll_task_free(). This is needed to avoid a rare error hit when running stress tests with chain DMA in user-space LL builds. Issue is hard to reproduce, but similar error signature can be created by passing K_NO_WAIT to k_sem_take() and running a test with chain-dma pipeline. Add defensive code that handles this scenario and prints out a warning when unexpected return occurs. Tested with a custom build with K_NO_WAIT passed to k_sem_take(). Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
65f092d to
d01c749
Compare
|
V36 update:
|
|
V36:
|
There was a problem hiding this comment.
Pull request overview
This PR is an early-stage step toward running low-latency (LL) audio pipeline work in Zephyr user-space by making pipeline position tracking safe/accessible from user threads, and by enabling the required Zephyr userspace configuration on ACE30 boards.
Changes:
- Move
pipeline_posn_init()earlier in the Zephyr init sequence and add user-thread access grants for the pipeline position mutex. - Update pipeline position locking to use a mutex (userspace-safe) when
CONFIG_SOF_USERSPACE_LLis enabled, and avoid copying non-userspace-accessible trace context. - Enable/adjust userspace LL-related Kconfig settings in ACE30 PTL/WCL board configs and disable features noted as not yet userspace-compatible.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| zephyr/wrapper.c | Removes late pipeline position init from task_main_start() (init is moved earlier). |
| src/schedule/zephyr_ll.c | Adds warning/handling when task-free semaphore wait times out. |
| src/ipc/ipc-common.c | Grants IPC userspace thread access to the pipeline position mutex. |
| src/init/init.c | Calls pipeline_posn_init() earlier (before platform_init() / userspace IPC init). |
| src/include/sof/audio/pipeline.h | Declares pipeline_posn_grant_access() for userspace LL builds. |
| src/include/sof/audio/pipeline-trace.h | Adds a build-time guard for userspace LL + non-Zephyr-log trace usage. |
| src/audio/pipeline/pipeline-graph.c | Uses mutex-based locking for pipeline position offsets under userspace LL and adds access-grant helper. |
| app/boards/intel_adsp_ace30_wcl.conf | Enables userspace + userspace LL and disables currently incompatible features. |
| app/boards/intel_adsp_ace30_ptl.conf | Enables userspace LL and disables currently incompatible features. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| if (wait_ret && task->state != SOF_TASK_STATE_FREE) { | ||
| tr_warn(&ll_tr, "task %p still active on free (state %d, semret %d)", | ||
| task, task->state, wait_ret); | ||
| zephyr_ll_task_done(sch, task); | ||
| } |
|
V37 version:
|
The direction-of-arrival emphasis coefficient tables iir_emphasis_48k and iir_emphasis_16k were declared as mutable globals, so the linker placed them in the .data section. The tables are read-only coefficient data, never modified, so mark them static const. This mirrors how sound_dose stores its IIR coefficient table. As another benefit, this change makes it possible to run tdfb in user-space as user threads can access rodata. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Make ipc_msg_send() a Zephyr system call so audio processing modules running in user-space LL threads can queue IPC messages (e.g. position updates, notifications) back to the host. The change takes effect only if CONFIG_SOF_USERSPACE_LL=y. Follows the same pattern used for ipc_msg_reply(): a dedicated header with __syscall declaration, z_impl/z_vrfy split, and syscall header registration in CMakeLists.txt. The verifier validates that the msg struct is writable (the implementation touches the list linkage) and that the data buffer, when provided, is readable up to msg->tx_size bytes. Add ipc_msg_list_remove() as a system call that acquires the IPC lock and removes a message from its list. This allows message freeing to work from userspace threads that cannot access kernel spinlocks directly. Update ipc_msg_free() in msg.h to use ipc_msg_list_remove() instead of directly accessing the IPC spinlock. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com> (cherry picked from commit 0161f70)
Add an optional heap parameter to ipc_msg_w_ext_init() and ipc_msg_init() so callers can direct allocations to a specific heap. When the heap argument is NULL the existing rzalloc() path is used; when non-NULL, sof_heap_alloc()/sof_heap_free() are used instead. This allows IPC messages to be allocated from userspace-accessible heaps. For audio module contexts, introduce mod_ipc_msg_w_ext_init() and mod_ipc_msg_init() in generic.h. These use mod_zalloc()/ mod_free() for allocations that are automatically tracked and freed with the module lifecycle. ipc_msg_w_ext_init() is moved from a static inline in msg.h to a non-inline function in ipc-common.c due to the additional sof_heap_alloc dependency. Update all existing callers: - Module context callers (cadence, sound_dose, tdfb, mfcc) use the new mod_ipc_msg_*() variants and mod_ipc_msg_free(). - host-zephyr.c uses hd->heap, pipeline-graph.c uses the heap parameter from pipeline_new(). - Remaining kernel-context callers pass NULL for the default heap. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com> (cherry picked from commit 338b28e)
Move user-facing notification functions (send_copier_gateway_xrun_notif_msg, send_gateway_xrun_notif_msg, send_mixer_underrun_notif_msg, send_process_data_error_notif_msg) to a new notification-user.c file so they can run in userspace. The send_resource_notif() function, which depends on the kernel-side notification pool and IPC message infrastructure, is converted to a Zephyr syscall. The implementation is renamed to z_impl_send_resource_notif() and remains in notification.c alongside is_notif_filtered_out() and ipc4_update_notification_mask(). The send_resource_notif() is converted to a system call only if CONFIG_SOF_USERSPACE_LL=y, without it the behaviour is same as befofe. A z_vrfy_send_resource_notif() handler is added to validate the user-provided data buffer and other parameters before forwarding to the kernel implementation. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com> (cherry picked from commit fff3d44)
Bump the LL heap size of 0x8000 (512 KiB). This is current maximum with CONFIG_VIRTUAL_HEAP (enabled by default for PTL). Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Pull in in-review PR to enable testing topologies that use 4ch PCH DMIC configuration on Intel PTL. Link: zephyrproject-rtos/zephyr#117022 Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
|
V38 update:
|
SOF has recently gained ability to run DP (=preemptable audio tasks) in Zephyr user-space.
This PR is an early stage pull-request for changes to extend this capability to all of the audio pipeline code, and specifically the LL (low-latency) tasks.
This early stage as the design is not set in stone and the PR uses a number of short cuts in order to move (and tests) incrementally larger sets of audio functionality.