Skip to content

Commit c918f15

Browse files
mjguzikbrauner
authored andcommitted
fs: call inode_sb_list_add() outside of inode hash lock
As both locks are highly contended during significant inode churn, holding the inode hash lock while waiting for the sb list lock exacerbates the problem. Why moving it out is safe: the inode at hand still has I_NEW set and anyone who finds it through legitimate means waits for the bit to clear, by which time inode_sb_list_add() is guaranteed to have finished. This significantly drops hash lock contention for me when stating 20 separate trees in parallel, each with 1000 directories * 1000 files. However, no speed up was observed as contention increased on the other locks, notably dentry LRU. Even so, removal of the lock ordering will help making this faster later. Signed-off-by: Mateusz Guzik <mjguzik@gmail.com> Link: https://lore.kernel.org/r/20250320004643.1903287-1-mjguzik@gmail.com Reviewed-by: Jan Kara <jack@suse.cz> Signed-off-by: Christian Brauner <brauner@kernel.org>
1 parent d5a05a5 commit c918f15

1 file changed

Lines changed: 5 additions & 5 deletions

File tree

fs/inode.c

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1300,8 +1300,8 @@ struct inode *inode_insert5(struct inode *inode, unsigned long hashval,
13001300
}
13011301

13021302
if (set && unlikely(set(inode, data))) {
1303-
inode = NULL;
1304-
goto unlock;
1303+
spin_unlock(&inode_hash_lock);
1304+
return NULL;
13051305
}
13061306

13071307
/*
@@ -1313,14 +1313,14 @@ struct inode *inode_insert5(struct inode *inode, unsigned long hashval,
13131313
hlist_add_head_rcu(&inode->i_hash, head);
13141314
spin_unlock(&inode->i_lock);
13151315

1316+
spin_unlock(&inode_hash_lock);
1317+
13161318
/*
13171319
* Add inode to the sb list if it's not already. It has I_NEW at this
13181320
* point, so it should be safe to test i_sb_list locklessly.
13191321
*/
13201322
if (list_empty(&inode->i_sb_list))
13211323
inode_sb_list_add(inode);
1322-
unlock:
1323-
spin_unlock(&inode_hash_lock);
13241324

13251325
return inode;
13261326
}
@@ -1449,8 +1449,8 @@ struct inode *iget_locked(struct super_block *sb, unsigned long ino)
14491449
inode->i_state = I_NEW;
14501450
hlist_add_head_rcu(&inode->i_hash, head);
14511451
spin_unlock(&inode->i_lock);
1452-
inode_sb_list_add(inode);
14531452
spin_unlock(&inode_hash_lock);
1453+
inode_sb_list_add(inode);
14541454

14551455
/* Return the locked inode with I_NEW set, the
14561456
* caller is responsible for filling in the contents

0 commit comments

Comments
 (0)