Skip to content

Commit 3dae5c4

Browse files
nickdesaulniersingomolnar
authored andcommitted
x86/asm/bitops: Use __builtin_clz{l|ll} to evaluate constant expressions
Micro-optimize the bitops code some more, similar to commits: fdb6649 ("x86/asm/bitops: Use __builtin_ctzl() to evaluate constant expressions") 2fcff79 ("powerpc: Use builtin functions for fls()/__fls()/fls64()") From a recent discussion, I noticed that x86 is lacking an optimization that appears in arch/powerpc/include/asm/bitops.h related to constant folding. If you add a BUILD_BUG_ON(__builtin_constant_p(param)) to these functions, you'll find that there were cases where the use of inline asm pessimized the compiler's ability to perform constant folding resulting in runtime calculation of a value that could have been computed at compile time. Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: https://lore.kernel.org/r/20230828-x86_fls-v1-1-e6a31b9f79c3@google.com
1 parent 4accdb9 commit 3dae5c4

1 file changed

Lines changed: 9 additions & 0 deletions

File tree

arch/x86/include/asm/bitops.h

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -293,6 +293,9 @@ static __always_inline unsigned long variable_ffz(unsigned long word)
293293
*/
294294
static __always_inline unsigned long __fls(unsigned long word)
295295
{
296+
if (__builtin_constant_p(word))
297+
return BITS_PER_LONG - 1 - __builtin_clzl(word);
298+
296299
asm("bsr %1,%0"
297300
: "=r" (word)
298301
: "rm" (word));
@@ -360,6 +363,9 @@ static __always_inline int fls(unsigned int x)
360363
{
361364
int r;
362365

366+
if (__builtin_constant_p(x))
367+
return x ? 32 - __builtin_clz(x) : 0;
368+
363369
#ifdef CONFIG_X86_64
364370
/*
365371
* AMD64 says BSRL won't clobber the dest reg if x==0; Intel64 says the
@@ -401,6 +407,9 @@ static __always_inline int fls(unsigned int x)
401407
static __always_inline int fls64(__u64 x)
402408
{
403409
int bitpos = -1;
410+
411+
if (__builtin_constant_p(x))
412+
return x ? 64 - __builtin_clzll(x) : 0;
404413
/*
405414
* AMD64 says BSRQ won't clobber the dest reg if x==0; Intel64 says the
406415
* dest reg is undefined if x==0, but their CPU architect says its

0 commit comments

Comments
 (0)