Feature - SMP Granular Locks by sudeep-mohanty · Pull Request #1154 · FreeRTOS/FreeRTOS-Kernel

sudeep-mohanty · 2024-10-09T13:20:58Z

This PR adds support for granular locking to the FreeRTOS kernel.

Description

Granular locking introduces the concept of having localized locks per kernel data group for SMP configuration. This method is an optional replacement of the existing kernel locks and is controlled by a new port layer configuration, viz., portUSING_GRANULAR_LOCKS.

Test Steps

A Common/Minimal demo runner for Espressif targets is included in this PR's branch under FreeRTOS/Demo/Espressif_ESP32. The Check task polls every standard demo task on a 3-second cycle and prints per-test PASS / FAIL lines in a Unity-style format, ending with a totals line and an OK / FAILED verdict.

Resources

FreeRTOS granular locks kernel repository: https://github.com/sudeep-mohanty/FreeRTOS-Kernel/tree/feature/smp_granular_locks_v4
Common/Minimal demo project tree: https://github.com/sudeep-mohanty/FreeRTOS/tree/feature/smp_granular_locks_v4/FreeRTOS/Demo/Espressif_ESP32
Public ESP-IDF fork that tracks this work (includes a copy of the granular locks kernel): https://github.com/sudeep-mohanty/esp-idf/tree/feat/granular_locks_test

To run the demo on an ESP32

1. Clone the public ESP-IDF fork and set up the build environment:

git clone -b feat/granular_locks_test https://github.com/sudeep-mohanty/esp-idf.git
cd esp-idf
./install.sh
. ./export.sh

2. Build, flash, and monitor:

cd /path/to/FreeRTOS/FreeRTOS/Demo/Espressif_ESP32
idf.py set-target esp32 build flash monitor

3. Observe the output:

Per-cycle:

----- Cycle N (tick X) -----
test_MessageBuffer:PASS
test_TaskNotification:PASS
...
Cycle N:  PASS   FAIL

Final summary (after the configured runtime, default 180 s):

-----------------------
Demo run complete: N cycles, M s.

Per-test totals (PASS / FAIL over N cycles):
  test_MessageBuffer         N /     0
  ...
  test_AbortDelay            N /     0

 Tests  Failures
OK | FAILED
-----------------------

4. Configure the runtime (optional):

The run length is set by mainDEMO_DURATION_S (seconds; 0 runs until reset), overridden through the EXTRA_CFLAGS environment variable. EXTRA_CFLAGS is read at CMake configure time, so run idf.py fullclean first (or use a fresh build directory) when you change it.

EXTRA_CFLAGS="-DmainDEMO_DURATION_S=3600" idf.py build flash monitor   # run for 1 hour
EXTRA_CFLAGS="-DmainDEMO_DURATION_S=0"    idf.py build flash monitor   # run until reset

Test Results

Test	`portUSING_GRANULAR_LOCKS=1` `configNUMBER_OF_CORES=2` `configRUN_MULTIPLE_PRIORITIES=0`	`portUSING_GRANULAR_LOCKS=1` `configNUMBER_OF_CORES=2` `configRUN_MULTIPLE_PRIORITIES=1`
MessageBuffer	✅	✅
TaskNotification	✅	✅
EventGroup	✅	✅ 🔧
IntMath	✅	✅
QueuePeek	✅	✅ 🔧
BlockingQueue	✅	✅
SemTest	✅	✅
PollQueue	✅	✅
Flop	✅	✅
CountSem	✅	✅
Death	✅	✅
QueueOverwrite	✅	✅
IntStreamBuffer	✅	✅
MessageBufferAMP	✅	✅
QueueSet	✅ 🔧	✅ 🔧
QueueSetPolling	✅	✅
DynamicPriority	✅	✅ 🔧
AbortDelay	✅	✅ 🔧
StreamBuffer	✅ 🔧	✅
GenericQueue	✅ 🔧	✅ 🔧
IntSem	✅ 🔧	✅ 🔧
RecMutex	✅ 🔧	✅ 🔧
IntQueue	✅ 🔧	⚠️ 🔧
BlockTime	✅	✅ 🔧
StaticAllocation	✅	✅ 🔧
TimerDemo	✅ 🔧	✅ 🔧

The Common/Minimal test updates and the ESP32 demo these results were produced
with are in the companion PR FreeRTOS/FreeRTOS#1434.

Legend:

✅ — passes as-is on the Espressif_ESP32 demo runner
✅ 🔧 — passes after an upstream Common/Minimal test update; cell links to the commit in Make the Common/Minimal tests SMP safe, and add an Espressif ESP32 demo FreeRTOS#1434
⚠️ — passes, but has shown an intermittent failure under sustained load; see the note below
❌ — fails

⚠️ IntQueue under configRUN_MULTIPLE_PRIORITIES=1: a receiver can be starved
between xQueueReceive() removing a value from the queue and
prvRecordValue_NormallyEmpty() storing it, so the round-end scan sees it
missing. It ran clean for 44 h 2 m of the 48 h sweep before failing; single priority completed the full 48 h clean.

TODO

Publish test results
48 hour stress test — single priority PASS (57,610 cycles / 48 h / 1,497,860 passes / 0 failures); multi priority clean to 52,839 cycles (44 h 2 m) then the ⚠️ IntQueue latch above. No panics, reboots or backtraces in either run. Full results and per-test tallies
Publish granular-lock performance comparison results
Re-enable each ⛔ test under configRUN_MULTIPLE_PRIORITIES=1 (upstream Common/Minimal patches)

Checklist

I have tested my changes. No regression in existing tests.
I have modified and/or added unit-tests to cover the code changes in this Pull Request.

Related Issue

Closes Granular Lock for SMP #905

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

sudeep-mohanty · 2024-10-09T13:23:42Z

@chinglee-iot @aggarg This PR introduces the granular locks changes to the FreeRTOS kernel. Please have a look and we could have discussions/changes in this PR context. Thank you.

cc: @ESP-Marius @Dazza0

rawalexe · 2024-10-09T16:55:58Z

Thank you for your contribution, I'll forward this request to the team. There are few error in the PR can you please try fixing them

rawalexe · 2024-10-14T22:28:24Z

Hello @sudeep-mohanty, I am just following up if you had time to fix the build issues

sudeep-mohanty · 2024-10-15T08:26:38Z

@rawalexe Yes! I shall work on the failures and would also do some refactoring for an easier review process. For now, I've put this PR in draft.

ActoryOu · 2024-10-31T05:02:09Z

Hi @sudeep-mohanty,
Could you help check CI failing?

sudeep-mohanty · 2024-10-31T09:27:52Z

Hi @sudeep-mohanty, Could you help check CI failing?

Hi @ActoryOu, I've made some updates which should fix the CI failures however I could not understand why the link-verifier action fails. Seems more like a script failure to me. So this action would still fail. If you have more information on what is causing it, could you let me know and I shall fix it. Thanks.

sudeep-mohanty · 2024-11-01T10:57:39Z

Created a PR to fix the CI failure - FreeRTOS/FreeRTOS#1292

sonarqubecloud · 2024-11-04T15:38:43Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
6.3% Duplication on New Code

See analysis details on SonarCloud

chinglee-iot

Coding format and minor change suggestion.

chinglee-iot

Update const MISRA check
Update to use STATIC to align with other module
Update BASE_TYPE critical section logic
Update for readability.

…herit() xTaskPriorityInherit() is called inside a critical section from queue.c. This commit moves the critical section into xTaskPriorityInherit(). Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>

Changed xPreemptionDisable to be a count rather than a pdTRUE/pdFALSE. This allows nested calls to vTaskPreemptionEnable(), where a yield only occurs when xPreemptionDisable is 0. Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>

Adds the required checks for granular locking port macros. Port Config: - portUSING_GRANULAR_LOCKS to enable granular locks - portCRITICAL_NESTING_IN_TCB should be disabled Granular Locking Port Macros: - Spinlocks - portSPINLOCK_TYPE - portINIT_SPINLOCK( pxSpinlock ) - portINIT_SPINLOCK_STATIC - Locking - portGET_SPINLOCK() - portRELEASE_SPINLOCK() Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>

- Updated prvCheckForRunStateChange() for granular locks - Updated vTaskSuspendAll() and xTaskResumeAll() - Now holds the xTaskSpinlock during kernel suspension - Increments/decrements xPreemptionDisable. Only yields when 0, thus allowing for nested suspensions across different data groups Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>

Updated critical section macros with granular locks. Some tasks.c API relied on their callers to enter critical sections. This assumption no longer works under granular locking. Critical sections added to the following functions: - `vTaskInternalSetTimeOutState()` - `xTaskIncrementTick()` - `vTaskSwitchContext()` - `xTaskRemoveFromEventList()` - `vTaskInternalSetTimeOutState()` - `eTaskConfirmSleepModeStatus()` - `xTaskPriorityDisinherit()` - `pvTaskIncrementMutexHeldCount()` Added missing suspensions to the following functions: - `vTaskPlaceOnEventList()` - `vTaskPlaceOnUnorderedEventList()` - `vTaskPlaceOnEventListRestricted()` Fixed the locking in vTaskSwitchContext() vTaskSwitchContext() must aquire both kernel locks, viz., task lock and ISR lock. This is because, vTaskSwitchContext() can be called from either task context or ISR context. Also, vTaskSwitchContext() must not alter the interrupt state prematurely. Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>

Updated queue.c to use granular locking - Added xTaskSpinlock and xISRSpinlock - Replaced critical section macros with data group critical section macros such as taskENTER/EXIT_CRITICAL/_FROM_ISR() with queueENTER/EXIT_CRITICAL_FROM_ISR(). - Added vQueueEnterCritical/FromISR() and vQueueExitCritical/FromISR() which map to the data group critical section macros. - Added prvLockQueueForTasks() and prvUnlockQueueForTasks() as the granular locking equivalents to prvLockQueue() and prvUnlockQueue() respectively Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>

Updated event_groups.c to use granular locking - Added xTaskSpinlock and xISRSpinlock - Replaced critical section macros with data group critical section macros such as taskENTER/EXIT_CRITICAL/_FROM_ISR() with event_groupsENTER/EXIT_CRITICAL/_FROM_ISR(). - Added vEventGroupsEnterCritical/FromISR() and vEventGroupsExitCriti/FromISR() functions that map to the data group critical section macros. - Added prvLockEventGroupForTasks() and prvUnlockEventGroupForTasks() to suspend the event group when executing non-deterministic code. - xEventGroupSetBits() and vEventGroupDelete() accesses the kernel data group directly. Thus, added vTaskSuspendAll()/xTaskResumeAll() to these fucntions. Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>

Updated stream_buffer.c to use granular locking - Added xTaskSpinlock and xISRSpinlock - Replaced critical section macros with data group critical section macros such as taskENTER/EXIT_CRITICAL/_FROM_ISR() with sbENTER/EXIT_CRITICAL_FROM_ISR(). - Added vStreambuffersEnterCritical/FromISR() and vStreambuffersExitCritical/FromISR() to map to the data group critical section macros. - Added prvLockStreamBufferForTasks() and prvUnlockStreamBufferForTasks() to suspend the stream buffer when executing non-deterministic code. Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>

Updated timers.c to use granular locking - Added xTaskSpinlock and xISRSpinlock - Replaced critical section macros with data group critical section macros such as taskENTER/EXIT_CRITICAL() with tmrENTER/EXIT_CRITICAL(). - Added vTimerEnterCritical() and vTimerExitCritical() to map to the data group critical section macros. Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>

The design document captures the **Dual Spinlock With Data Group Locking** scheme used by the granular-locks implementation: data-group definitions and hierarchy, public and port-layer API, TCB locks, deferred state changes, ISR-only kernel critical-section helpers, event-list TOCTOU considerations, and current limitations.

sonarqubecloud · 2026-08-24T11:56:14Z

Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.5% Duplication on New Code

See analysis details on SonarQube Cloud

sudeep-mohanty marked this pull request as draft October 15, 2024 08:25

sudeep-mohanty force-pushed the feature/smp_granular_locks_v4 branch 9 times, most recently from 0159a4a to 5bf8c33 Compare October 30, 2024 10:26

sudeep-mohanty marked this pull request as ready for review October 30, 2024 10:34

sudeep-mohanty requested a review from a team as a code owner October 30, 2024 10:34

sudeep-mohanty force-pushed the feature/smp_granular_locks_v4 branch from 5bf8c33 to 9f8acc7 Compare October 31, 2024 09:25

sudeep-mohanty force-pushed the feature/smp_granular_locks_v4 branch 2 times, most recently from c79962d to f50d476 Compare October 31, 2024 12:43

sudeep-mohanty mentioned this pull request Nov 1, 2024

change(freertos/smp/granular_locks): Updated prvYieldCore() coverage tests FreeRTOS/FreeRTOS#1292

Closed

1 task

sudeep-mohanty force-pushed the feature/smp_granular_locks_v4 branch 4 times, most recently from 70eb466 to edc1c98 Compare November 4, 2024 15:38

sudeep-mohanty marked this pull request as draft November 15, 2024 14:25

chinglee-iot reviewed Dec 9, 2024

View reviewed changes

Comment thread timers.c Outdated

sudeep-mohanty changed the title ~~Feature/smp granular locks~~ Feature - SMP Granular Locks Jun 24, 2026

chinglee-iot reviewed Jul 21, 2026

View reviewed changes

Comment thread include/task.h Outdated

Comment thread include/task.h

Comment thread tasks.c Outdated

Comment thread tasks.c Outdated

Comment thread tasks.c Outdated

chinglee-iot reviewed Jul 22, 2026

View reviewed changes

Comment thread tasks.c Outdated

Comment thread tasks.c Outdated

sudeep-mohanty force-pushed the feature/smp_granular_locks_v4 branch from 5aede4a to 5860cfd Compare July 22, 2026 13:23

chinglee-iot reviewed Jul 27, 2026

View reviewed changes

Comment thread include/task.h Outdated

Comment thread queue.c Outdated

sudeep-mohanty force-pushed the feature/smp_granular_locks_v4 branch 3 times, most recently from 0bb306a to cff470e Compare August 6, 2026 13:24

sudeep-mohanty force-pushed the feature/smp_granular_locks_v4 branch from dd67c0a to 9decfa6 Compare August 17, 2026 09:30

This comment was marked as duplicate.

Sign in to view

chinglee-iot reviewed Aug 24, 2026

View reviewed changes

sudeep-mohanty marked this pull request as ready for review August 24, 2026 06:19

chinglee-iot reviewed Aug 24, 2026

View reviewed changes

Comment thread queue.c

Comment thread tasks.c

sudeep-mohanty force-pushed the feature/smp_granular_locks_v4 branch from 9decfa6 to b218d11 Compare August 24, 2026 10:55

Dazza0 and others added 11 commits August 24, 2026 13:41

refactor(freertos/smp): Move critical sections inside xTaskPriorityIn…

7710453

…herit() xTaskPriorityInherit() is called inside a critical section from queue.c. This commit moves the critical section into xTaskPriorityInherit(). Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>

feat(freertos/smp): Add support for TCB locks

c015a48

sudeep-mohanty force-pushed the feature/smp_granular_locks_v4 branch from b218d11 to 1132f5c Compare August 24, 2026 11:55

sudeep-mohanty mentioned this pull request Aug 31, 2026

Make the Common/Minimal tests SMP safe, and add an Espressif ESP32 demo FreeRTOS/FreeRTOS#1434

Open

2 tasks

Uh oh!

Conversation

sudeep-mohanty commented Oct 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Steps

Resources

To run the demo on an ESP32

Test Results

TODO

Checklist

Related Issue

Uh oh!

sudeep-mohanty commented Oct 9, 2024

Uh oh!

rawalexe commented Oct 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rawalexe commented Oct 14, 2024

Uh oh!

sudeep-mohanty commented Oct 15, 2024

Uh oh!

ActoryOu commented Oct 31, 2024

Uh oh!

sudeep-mohanty commented Oct 31, 2024

Uh oh!

sudeep-mohanty commented Nov 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sonarqubecloud Bot commented Nov 4, 2024

Quality Gate passed

Uh oh!

Uh oh!

chinglee-iot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as duplicate.

Uh oh!

This comment was marked as duplicate.

Uh oh!

This comment was marked as duplicate.

Uh oh!

chinglee-iot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sonarqubecloud Bot commented Aug 24, 2026

Quality Gate passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

sudeep-mohanty commented Oct 9, 2024 •

edited

Loading

rawalexe commented Oct 9, 2024 •

edited

Loading

sudeep-mohanty commented Nov 1, 2024 •

edited

Loading