mirrors/linux
0
0 Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git synced 2026-10-26 10:57:38 +08:00
Code Activity
Files
master
linux/mm/Kconfig.debug
T
Dave Hansen 47e725702b mm: make per-VMA locks available universally 
Patch series "mm: Unconditional per-VMA locks and cleanups", v7.

tl;dr: Make per-VMA locks available in all configs.  Simplify some of the
per-VMA lock users now that they can rely on them being always available.

Binder and networking folks: Your code is the target of the cleanups.  I'm
cc'ing you now on v2 because there's emerging consensus on the mm side
that the approach here is sane.  I'm not quite sure how this pile would
get merged, but ack/review tags would be appreciated if this looks good to
you.

Longer version:

When working on some x86 shadow stack code, it was a real pain to avoid
causing recursive locking problems with mmap_lock.  One way to avoid those
was to avoid mmap_lock and use per-VMA locks instead.  They are great, but
they are not available in all configs which makes them unusable in generic
code, or if you want to completely avoid mmap_lock.

Make per-VMA locks available in all configs.  Right now, they are only
available on select architectures when SMP and MMU are enabled.  But all
of the primitives that per-VMA locks are built on (RCU, maple trees,
refcounts) work just fine without SMP or MMU.

The only real downside is that making VMAs a wee bit bigger on !MMU and
!SMP builds.

The upside is much cleaner code, lower complexity and less #ifdeffery.

Clean up a binder VMA locking site now that it can rely on per-VMA locks.

Building on top of universally-available per-VMA locks, introduce a new
helper.  Since the new API does not require callers to have a fallback to
mmap_lock, it's much easier to use.  Callers can potentially replace this
very common kernel idiom:

	mmap_read_lock(mm);
	vma = vma_lookup()
	// fiddle with vma
	mmap_read_unlock(mm);

with:

	vma = vma_start_read_unlocked(mm, address);
	// fiddle with vma
	vma_end_read(vma);

Which avoids mmap_lock entirely in the fast path.

Use that new API for another binder site and one in the TCP code.


This patch (of 7):

The per-VMA locks have been around for several years.  They've had some
bugs worked out of them and have seen quite wide use.  However, they are
still only available when architectures explicitly enable them.  Remove
the conditional compilation around the per-VMA locks, making them
available on all architectures and configs.

The approach up to now seemed to be to add ARCH_SUPPORTS_PER_VMA_LOCK when
the architecture started using per-VMA locks in the fault handler.  But,
contrary to the naming, the Kconfig option does not really indicate
whether the architecture supports per-VMA locks or not.  It is more of a
marker for whether the architecture is likely to benefit from per-VMA
locks.

To me, the most important thing side-effect of universal availability is
letting per-VMA locks be used in SMP=n configs.  This lets us use
per-VMA locking in all x86 code without fallbacks.

Overall, this just generally makes the kernel simpler.  Just look at the
diffstat.  It also opens the door to users that want to use the per-VMA
locks in common code.  Doing *that* brings additional simplifications.

The downside of this is adding some fields to vm_area_struct and
mm_struct.  There are likely ways to optimize this, especially for things
like SMP=n configs.  For now, do the simplest thing: use the same
implementation everywhere.

== Considerations for NOMMU config ==

NOMMU systems do not write-lock VMAs, therefore read-locking a VMA would
always succeed unless VMA is detached.  Therefore for NOMMU config we make
vma_mark_attached() a NOOP, which keeps VMAs always in detached state. 
This causes VMA read-locking to always fail and the caller falls back to
locking mmap_lock.

The following functions will have a different implementation in NOMMU
config:

- vma_mark_attached(), vma_mark_detached() are made NOOPs, keeping VMAs
  always in a detached state and preventing assertions and refcount
  underflows;

- vma_start_write(), vma_start_write_killable() are made NOOPs to avoid
  warnings in __vma_start_write() due to VMAs being detached.  These
  functions are not used in NOMMU code but __vma_start_write() is an
  exported function, therefore might be used by drivers.

- vma_assert_attached() is made NOOP because it's reachable from NOMMU
  code via split_vma()->vma_iter_store_new()->vma_iter_store_overwrite();

- vma_assert_write_locked() is asserting vma->vm_mm is write-locked, as
  was done before this change;

- vma_assert_locked() is asserting vma->vm_mm is locked, as was done
  before this change;

The following functions work for both MMU and NOMMU configs:

- vma_lock_init() performs the same initialization as for MMU config;

- mm_lock_seqcount_init(), mm_lock_seqcount_begin(),
  mm_lock_seqcount_end() are called from mmap_write_{lock|unlock} and
  update mm_lock_seq correctly.

- mmap_lock_speculate_try_begin(), mmap_lock_speculate_retry() work as
  is because mm_lock_seq is updated correctly;

- vma_start_read(), vma_start_read_locked() will always fail because
  VMAs are always detached;

- vma_end_read() will never be called because vma_start_read() never
  succeeds;

- vma_is_attached() always return false because VMAs are always
  detached;

- vma_assert_detached() will never trigger because VMAs are never
  attached;

- vma_start_read_locked() always return false because VMAs are always
  detached;

- lock_vma_under_rcu() will be safe as the attempted read lock will bail;

Changes in the following files are not affecting NOMMU config:

task_mmu.c - not compiled when CONFIG_MMU=n;
pagewalk.c - not compiled when CONFIG_MMU=n;
userfaultfd.c - not compiled when CONFIG_MMU=n (CONFIG_USERFAULTFD depends
on CONFIG_MMU);

The following changes in the BPF code are made to keep NOMMU config
working like before:

stack_map_lock_vma() - keeps mmap_lock in NOMMU config;
bpf_iter_task_vma_new() - bails out in NOMMU config;

Link: https://lore.kernel.org/20260831203056.838265-1-surenb@google.com
Link: https://lore.kernel.org/20260831203056.838265-2-surenb@google.com
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Todd Kjos <tkjos@android.com>
Cc: Christian Brauner <christian@brauner.io>
Cc: Carlos Llamas <cmllamas@google.com>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: David Ahern <dsahern@kernel.org>
Cc: Arve Hjønnevåg <arve@android.com>
2026-10-06 17:39:10 -07:00

349 lines
12 KiB
Plaintext
Raw Permalink Blame History

# SPDX-License-Identifier: GPL-2.0-only
config PAGE_EXTENSION
bool "Extend memmap on extra space for more information on page"
help
Extend memmap on extra space for more information on page. This
could be used for debugging features that need to insert extra
field for every page. This extension enables us to save memory
by not allocating this extra memory according to boottime
configuration.
config DEBUG_PAGEALLOC
bool "Debug page memory allocations"
depends on DEBUG_KERNEL
depends on !HIBERNATION || ARCH_SUPPORTS_DEBUG_PAGEALLOC && !PPC && !SPARC
select PAGE_POISONING if !ARCH_SUPPORTS_DEBUG_PAGEALLOC
help
Unmap pages from the kernel linear mapping after free_pages().
Depending on runtime enablement, this results in a small or large
slowdown, but helps to find certain types of memory corruption.
Also, the state of page tracking structures is checked more often as
pages are being allocated and freed, as unexpected state changes
often happen for same reasons as memory corruption (e.g. double free,
use-after-free). The error reports for these checks can be augmented
with stack traces of last allocation and freeing of the page, when
PAGE_OWNER is also selected and enabled on boot.
For architectures which don't enable ARCH_SUPPORTS_DEBUG_PAGEALLOC,
fill the pages with poison patterns after free_pages() and verify
the patterns before alloc_pages(). Additionally, this option cannot
be enabled in combination with hibernation as that would result in
incorrect warnings of memory corruption after a resume because free
pages are not saved to the suspend image.
By default this option will have a small overhead, e.g. by not
allowing the kernel mapping to be backed by large pages on some
architectures. Even bigger overhead comes when the debugging is
enabled by DEBUG_PAGEALLOC_ENABLE_DEFAULT or the debug_pagealloc
command line parameter.
config DEBUG_PAGEALLOC_ENABLE_DEFAULT
bool "Enable debug page memory allocations by default?"
depends on DEBUG_PAGEALLOC
help
Enable debug page memory allocations by default? This value
can be overridden by debug_pagealloc=off|on.
config SLUB_DEBUG
default y
bool "Enable SLUB debugging support" if EXPERT
depends on SYSFS && !SLUB_TINY
select STACKDEPOT if STACKTRACE_SUPPORT
help
SLUB has extensive debug support features. Disabling these can
result in significant savings in code size. While /sys/kernel/slab
will still exist (with SYSFS enabled), it will not provide e.g. cache
validation.
config SLUB_DEBUG_ON
bool "SLUB debugging on by default"
depends on SLUB_DEBUG
select STACKDEPOT_ALWAYS_INIT if STACKTRACE_SUPPORT
default n
help
Boot with debugging on by default. SLUB boots by default with
the runtime debug capabilities switched off. Enabling this is
equivalent to specifying the "slab_debug" parameter on boot.
There is no support for more fine grained debug control like
possible with slab_debug=xxx. SLUB debugging may be switched
off in a kernel built with CONFIG_SLUB_DEBUG_ON by specifying
"slab_debug=-".
config SLUB_RCU_DEBUG
bool "Enable UAF detection in TYPESAFE_BY_RCU caches (for KASAN)"
depends on SLUB_DEBUG
# SLUB_RCU_DEBUG should build fine without KASAN, but is currently useless
# without KASAN, so mark it as a dependency of KASAN for now.
depends on KASAN
default KASAN_GENERIC || KASAN_SW_TAGS
help
Make SLAB_TYPESAFE_BY_RCU caches behave approximately as if the cache
was not marked as SLAB_TYPESAFE_BY_RCU and every caller used
kfree_rcu() instead.
This is intended for use in combination with KASAN, to enable KASAN to
detect use-after-free accesses in such caches.
(KFENCE is able to do that independent of this flag.)
This might degrade performance.
Unfortunately this also prevents a very specific bug pattern from
triggering (insufficient checks against an object being recycled
within the RCU grace period); so this option can be turned off even on
KASAN builds, in case you want to test for such a bug.
If you're using this for testing bugs / fuzzing and care about
catching all the bugs WAY more than performance, you might want to
also turn on CONFIG_RCU_STRICT_GRACE_PERIOD.
WARNING:
This is designed as a debugging feature, not a security feature.
Objects are sometimes recycled without RCU delay under memory pressure.
If unsure, say N.
config PAGE_OWNER
bool "Track page owner"
depends on DEBUG_KERNEL && STACKTRACE_SUPPORT
select DEBUG_FS
select STACKTRACE
select STACKDEPOT
select PAGE_EXTENSION
help
This keeps track of what call chain is the owner of a page, may
help to find bare alloc_page(s) leaks. Even if you include this
feature on your build, it is disabled in default. You should pass
"page_owner=on" to boot parameter in order to enable it. Eats
a fair amount of memory if enabled. See tools/mm/page_owner_sort.c
for user-space helper.
If unsure, say N.
config PAGE_TABLE_CHECK
bool "Check for invalid mappings in user page tables"
depends on ARCH_SUPPORTS_PAGE_TABLE_CHECK
depends on EXCLUSIVE_SYSTEM_RAM
select PAGE_EXTENSION
help
Check that anonymous page is not being mapped twice with read write
permissions. Check that anonymous and file pages are not being
erroneously shared. Since the checking is performed at the time
entries are added and removed to user page tables, leaking, corruption
and double mapping problems are detected synchronously.
If unsure say "n".
config PAGE_TABLE_CHECK_ENFORCED
bool "Enforce the page table checking by default"
depends on PAGE_TABLE_CHECK
help
Always enable page table checking. By default the page table checking
is disabled, and can be optionally enabled via page_table_check=on
kernel parameter. This config enforces that page table check is always
enabled.
If unsure say "n".
config PAGE_POISONING
bool "Poison pages after freeing"
help
Fill the pages with poison patterns after free_pages() and verify
the patterns before alloc_pages. The filling of the memory helps
reduce the risk of information leaks from freed data. This does
have a potential performance impact if enabled with the
"page_poison=1" kernel boot option.
Note that "poison" here is not the same thing as the "HWPoison"
for CONFIG_MEMORY_FAILURE. This is software poisoning only.
If you are only interested in sanitization of freed pages without
checking the poison pattern on alloc, you can boot the kernel with
"init_on_free=1" instead of enabling this.
If unsure, say N
config DEBUG_PAGE_REF
bool "Enable tracepoint to track down page reference manipulation"
depends on DEBUG_KERNEL
depends on TRACEPOINTS
help
This is a feature to add tracepoint for tracking down page reference
manipulation. This tracking is useful to diagnose functional failure
due to migration failures caused by page reference mismatches. Be
careful when enabling this feature because it adds about 30 KB to the
kernel code. However the runtime performance overhead is virtually
nil until the tracepoints are actually enabled.
config DEBUG_RODATA_TEST
bool "Testcase for the marking rodata read-only"
depends on STRICT_KERNEL_RWX
help
This option enables a testcase for the setting rodata read-only.
config ARCH_HAS_DEBUG_WX
bool
config DEBUG_WX
bool "Warn on W+X mappings at boot"
depends on ARCH_HAS_DEBUG_WX
depends on ARCH_HAS_PTDUMP
depends on MMU
select PTDUMP
help
Generate a warning if any W+X mappings are found at boot.
This is useful for discovering cases where the kernel is leaving W+X
mappings after applying NX, as such mappings are a security risk.
Look for a message in dmesg output like this:
<arch>/mm: Checked W+X mappings: passed, no W+X pages found.
or like this, if the check failed:
<arch>/mm: Checked W+X mappings: failed, <N> W+X pages found.
Note that even if the check fails, your kernel is possibly
still fine, as W+X mappings are not a security hole in
themselves, what they do is that they make the exploitation
of other unfixed kernel bugs easier.
There is no runtime or memory usage effect of this option
once the kernel has booted up - it's a one time check.
If in doubt, say "Y".
config ARCH_HAS_PTDUMP
bool
config PTDUMP
bool
config PTDUMP_DEBUGFS
bool "Export kernel pagetable layout to userspace via debugfs"
depends on DEBUG_KERNEL
depends on DEBUG_FS
depends on ARCH_HAS_PTDUMP
select PTDUMP
help
Say Y here if you want to show the kernel pagetable layout in a
debugfs file. This information is only useful for kernel developers
who are working in architecture specific areas of the kernel.
It is probably not a good idea to enable this feature in a production
kernel.
If in doubt, say N.
config HAVE_DEBUG_KMEMLEAK
bool
config DEBUG_KMEMLEAK
bool "Kernel memory leak detector"
depends on DEBUG_KERNEL && HAVE_DEBUG_KMEMLEAK
select DEBUG_FS
select STACKTRACE if STACKTRACE_SUPPORT
select KALLSYMS
select CRC32
select STACKDEPOT
select STACKDEPOT_ALWAYS_INIT if !DEBUG_KMEMLEAK_DEFAULT_OFF
help
Say Y here if you want to enable the memory leak
detector. The memory allocation/freeing is traced in a way
similar to the Boehm's conservative garbage collector, the
difference being that the orphan objects are not freed but
only shown in /sys/kernel/debug/kmemleak. Enabling this
feature will introduce an overhead to memory
allocations. See Documentation/dev-tools/kmemleak.rst for more
details.
Enabling SLUB_DEBUG may increase the chances of finding leaks
due to the slab objects poisoning.
In order to access the kmemleak file, debugfs needs to be
mounted (usually at /sys/kernel/debug).
config DEBUG_KMEMLEAK_MEM_POOL_SIZE
int "Kmemleak memory pool size"
depends on DEBUG_KMEMLEAK
range 200 1000000
default 16000
help
Kmemleak must track all the memory allocations to avoid
reporting false positives. Since memory may be allocated or
freed before kmemleak is fully initialised, use a static pool
of metadata objects to track such callbacks. After kmemleak is
fully initialised, this memory pool acts as an emergency one
if slab allocations fail.
config DEBUG_KMEMLEAK_DEFAULT_OFF
bool "Default kmemleak to off"
depends on DEBUG_KMEMLEAK
help
Say Y here to disable kmemleak by default. It can then be enabled
on the command line via kmemleak=on.
config DEBUG_KMEMLEAK_AUTO_SCAN
bool "Enable kmemleak auto scan thread on boot up"
default y
depends on DEBUG_KMEMLEAK
help
Depending on the cpu, kmemleak scan may be cpu intensive and can
stall user tasks at times. This option enables/disables automatic
kmemleak scan at boot up.
Say N here to disable kmemleak auto scan thread to stop automatic
scanning. Disabling this option disables automatic reporting of
memory leaks.
If unsure, say Y.
config DEBUG_KMEMLEAK_VERBOSE
bool "Default kmemleak to verbose mode"
depends on DEBUG_KMEMLEAK_AUTO_SCAN
help
Say Y here to have kmemleak print unreferenced object details
(backtrace, hex dump, address) to dmesg when new memory leaks are
detected during automatic scanning. This can also be toggled at
runtime via /sys/module/kmemleak/parameters/verbose.
If unsure, say N.
config PER_VMA_LOCK_STATS
bool "Statistics for per-vma locks"
help
Say Y here to enable success, retry and failure counters of page
faults handled under protection of per-vma locks. When enabled, the
counters are exposed in /proc/vmstat. This information is useful for
kernel developers to evaluate effectiveness of per-vma locks and to
identify pathological cases. Counting these events introduces a small
overhead in the page fault path.
If in doubt, say N.
config MEM_ALLOC_PROFILING
bool "Enable memory allocation profiling"
default n
depends on MMU
depends on PROC_FS
select CODE_TAGGING
select PAGE_EXTENSION
select SLAB_OBJ_EXT
help
Track allocation source code and record total allocation size
initiated at that code location. The mechanism can be used to track
memory leaks with a low performance and memory impact.
config MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT
bool "Enable memory allocation profiling by default"
default y
depends on MEM_ALLOC_PROFILING
config MEM_ALLOC_PROFILING_DEBUG
bool "Memory allocation profiler debugging"
default n
depends on MEM_ALLOC_PROFILING
select MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT
help
Adds warnings with helpful error messages for memory allocation
profiling.
View Git Blame Copy Permalink