mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2024-11-21 19:46:16 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

debugobjects: Refill per CPU pool more agressively

Browse Source 
Right now the per CPU pools are only refilled when they become
empty. That's suboptimal especially when there are still non-freed objects
in the to free list.

Check whether an allocation from the per CPU pool emptied a batch and try
to allocate from the free pool if that still has objects available.

   	    kmem_cache_alloc()	kmem_cache_free()
Baseline:   295k		245k
Refill:	    225k		173k

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Zhen Lei <thunder.leizhen@huawei.com>
Link: https://lore.kernel.org/all/20241007164914.439053085@linutronix.de
This commit is contained in:
Thomas Gleixner 2024-10-07 18:50:20 +02:00
parent a201a96b96
commit 13f9ca7239
1 changed files with 18 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
lib/debugobjects.c
View File

@ -255,6 +255,24 @@ static struct debug_obj *pcpu_alloc(void)
if (likely(obj)) {
pcp->cnt--;
/*
* If this emptied a batch try to refill from the
* free pool. Don't do that if this was the top-most
* batch as pcpu_free() expects the per CPU pool
* to be less than ODEBUG_POOL_PERCPU_SIZE.
*/
if (unlikely(pcp->cnt < (ODEBUG_POOL_PERCPU_SIZE - ODEBUG_BATCH_SIZE) &&
!(pcp->cnt % ODEBUG_BATCH_SIZE))) {
/*
* Don't try to allocate from the regular pool here
* to not exhaust it prematurely.
*/
if (pool_count(&pool_to_free)) {
guard(raw_spinlock)(&pool_lock);
pool_move_batch(pcp, &pool_to_free);
pcpu_refill_stats();
}
}
return obj;
}