mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
synced 2026-10-26 10:57:38 +08:00
master
189
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Linus Torvalds
|
55fbedcec6 |
Merge tag 'bpf-next-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
Pull bpf updates from Alexei Starovoitov:
"Major changes:
- Introduce scalar evolution (SCEV) analysis in the verifier.
It infers how registers and stack slots change in a loop and lets
the verifier check bounded loops with induction variables as ranges
instead of walking every iteration. The number of processed insns
drops ten fold in programs with loops (Eduard Zingerman)
- Add 'callx' instruction: indirect call of a static BPF subprogram.
The address is taken with ld_imm64 or loaded from read-only data.
Supported by x86-64, arm64 and riscv64 JITs (Alexei Starovoitov, Pu
Lehui)
- Add unwind_goto and unwind_exit instructions, so that Rust panic or
C++ exception can run destructors of every frame it passes through.
The normal path is free of unwind code. Supported by x86-64 and
arm64 JITs (Alexei Starovoitov)
- Support programs compiled by rust-bpf: treat load and store through
a number as arena access with BPF_F_ARENA_SCALAR, accept rust names
in BTF (Alexei Starovoitov)
- Raise BPF program stack size from 512 bytes to 2 KiB on x86-64 and
arm64 (Kumar Kartikeya Dwivedi)
- Support struct, union and __int128 up to 16 bytes passed and
returned by value in BPF subprograms and kfuncs, and fix x86-64 and
arm64 trampolines for such arguments (Yonghong Song)
- Introduce a common way to attach struct_ops to a cgroup via
bpf_link. bpf_tcp_ops is the first user and mirrors TCP sock_ops
callbacks (Martin KaFai Lau, Amery Hung)
- Describe inlined functions in BTF. New kinds tell where the inline
sites are and how to fetch the arguments there.
CONFIG_DEBUG_INFO_BTF_INLINE keeps it apart from the main BTF in
.BTF.inline section or in btf_vmlinux_inline.ko (Alan Maguire)
- Fix many memory safety, lifetime and locking bugs in BPF core,
verifier, iterators, local storage, dynptr, devmap, cpumap and
seg6local (Josef Bacik, Alexei Starovoitov)
- Diff stat is dominated by selftests and by removal of
lib/test_bpf.c. Its 16k lines of instruction tests were converted
to BPF selftests that BPF CI runs on every architecture it covers
(Daniel Borkmann)
Other features and fixes:
- Add KASAN checks to memory accesses of JITed programs on x86-64
(Alexis Lothoré)
- Complete unification of helper, kfunc and subprog argument checks
in the verifier (Amery Hung)
- Add bpf_copy_from_user_mm[_str]() kfuncs (Anastasios Papagiannis)
- Various features and fixes (Daniel Borkmann):
- Add .bpf keyring for program signature verification, modelled
after dm-verity keyring, and support ML-DSA in signing tools
- Emit shorter and faster code in x86-64 JIT for patterns that
LLVM generates
- Fix various issues in gotox handling
- Let BPF LSM set xattrs at inode creation with bpf_inode_init_xattr()
(David Windsor, Daniel Borkmann)
- Use sleepable page allocation in sleepable arena paths. Fix races
between arena free, allocation and user page fault (Emil Tsalapatis)
- Switch BPF crypto from lskcipher to AES-CBC and AES-ECB libraries
(Eric Biggers)
- Add bpf_proactive_reclaim() kfunc (Hui Zhu)
- Render decl_tags in libbpf btf_dump and add tests for bpftool C
dump (Ihor Solodrai)
- Speed up tracing_multi attach on x86 (Jiri Olsa)
- Various features and fixes (Kumar Kartikeya Dwivedi):
- Add BPF_PROG_STREAM_OPEN command to read BPF streams through a
file descriptor with poll support
- Let kernel structs publish arena pointers with __arena type tag
- Fix __uninit kfunc output buffers and fastcall rewrite with
indirect stack accesses
- Refuse CO-RE relocations for signed programs
- Add enqueue_rcvq and dequeue_rcvq hooks to bpf_tcp_ops to adjust
sk_rcvlowat and avoid unnecessary wakeups (Kuniyuki Iwashima)
- Add bpf_ksock_sendto() kfunc and allow ksock connect and send from
bpf_wq (Mahe Tardy)
- Fix racy plt target update in arm64 bpf_arch_text_poke()
(Matthew Wood)
- Restructure BPF selftests Makefile as a layered build
(Mykola Lysenko)
- Speed up hash map lookups for u32 and u64 keys (Mykyta Yatsenko)
- Add signed div/mod, movsx, bswap and gotol to MIPS JITs, signed
div/mod to parisc JITs (Nicholas Dudar, Johan Almbladh)
- Fix stack overflow in build ID parser on 32-bit (Nickolai
Zeldovich)
- Add stack arguments, signed arena loads and gotox to riscv64 JIT.
Pu Lehui is now co-maintainer of riscv BPF JITs (Pu Lehui, Chen
Pei, Feng Jiang)
- Add bpf_call_rcu() and bpf_call_rcu_tasks_trace() kfuncs. Support
fetching and/or/xor atomics in arena on x86-64 (Puranjay Mohan)
- Fix flaky selftests, selftests with clang 23 and bpf-gcc
(Qiliang Yuan, KaFai Wan, Vineet Gupta)
- Fix builds broken by fatal resolve_btfids warnings, BTF generation
on ARC, and define arch_cmpxchg_local() on arc, hexagon and
openrisc (Sasha Levin)
- Reduce memory use of hash maps by up to 50% per element
(T.J. Mercier)
- Mark pointer fields of sched_ext ops arguments as trusted (Tejun
Heo)
- Remove legacy bpf_asm, bpf_dbg and bpf_jit_disasm tools (Xia Yan)
- Fix index overflow in queue and stack maps, arena range tree after
allocation failure, tail call loop after freplace detach, lost
signal in bpf_send_signal() (Yuan Chen)
- Many more fixes and cleanups"
* tag 'bpf-next-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (660 commits)
selftests/bpf: Time tc_edt from its first packet with data
selftests/bpf: Test unwind_exit of an exception callback
selftests/bpf: Tests for 32-bit scalar evolution bounds computation
bpf: Reject unwind_goto and unwind_exit in offloaded progs
bpf: Support 32-bit linear scalar evolution
bpf: Make unwind_exit of an exception callback exit in JITs
selftests/bpf: Test forbidden bpf_ksock_sendto() LSM attach
selftests/bpf: Test ksock sendto()
bpf: Add bpf_ksock_sendto() kfunc
selftests/bpf: Test unwinding from R0: R2 subprog
selftests/bpf: Add a C++ prog with exceptions
selftests/bpf: Add tests for unwind_goto and unwind_exit
bpf, arm64: Support unwind_goto and unwind_exit
bpf, x86: Support unwind_goto and unwind_exit
bpf: Add unwind_goto and unwind_exit insns
tools/bpf: Remove legacy BPF tools
bpf: Annotate data races on bpf_lru_node type
selftests/bpf: Add arena allocation race tests
bpf: Fix arena race between page free and alloc leading to incoherency
bpf: Revalidate faulted arena pages before installing user mappings
...
|
||
|
Alan Maguire
|
672981f4ae |
kbuild, btf: Expose vmlinux inline BTF through sysfs
Expose BTF inline-location information from vmlinux .BTF.inline sections in /sys/kernel/btf as vmlinux.inline. Embedded vmlinux inline BTF supports read-only mmap(). Provide runtime support for a module-delivered vmlinux inline BTF carrier. Its link record binds the carrier module name, BTF size, and SHA-256 digest to the running kernel before its data is made available through a lazy vmlinux.inline sysfs attribute. The carrier is handled before ordinary module BTF registration, so its incidental .BTF data is not exposed. Make CONFIG_DEBUG_INFO_BTF_INLINE tristate. When configured as a module, retain vmlinux final split inline BTF for btf_vmlinux_inline.ko instead of embedding it in vmlinux. Reserve .BTF.inline.link in vmlinux and extend the final resolve_btfids patch pass to bind btf_vmlinux_inline to the raw vmlinux.BTF.inline payload. This preserves the normal built-in configuration while allowing the large inline BTF payload to load on demand. Normal module inline BTF is handled separately, after its references have been relocated against the final module BTF. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261009112324.1442319-4-alan.maguire@oracle.com |
||
|
Lorenzo Stoakes (ARM)
|
531980b4b1 |
kallsyms: reimplement mksysmap in C
mksysmap is a sed script consisting of 30 patterns which link-vmlinux.sh
uses to generate *.syms files, and which kallsyms is then called against to
generate *.kallsyms files, with the final vmlinux build ultimately
generating System.map.
For an x86-64 allmodconfig build, this involves three nm runs over a 250
MiB file and half a million lines written and read each time - 0.5s per
pass for llvm-nm, and 0.2s for GNU nm, with parsing on top of that.
This is unnecessary, instead have kallsyms simply read the ELF file
directly making use of the existing elf-parse library in scripts/.
This changes kallsyms such that its input is no longer the output from nm,
but rather an ELF file.
However, if the input file is empty, it outputs an empty table, which
retains the same behaviour on first pass that the build system expects.
System.map is byte-identical to nm | mksysmap for GNU nm and llvm-nm on
two x86 configurations each, and for llvm-nm on arm64, arm, s390 and
loongarch defconfigs, so are the kallsyms tables of every pass.
Relinking vmlinux, link steps included:
before after
allmodconfig clang 9.1s 7.9s
allmodconfig gcc 7.8s 7.6s
defconfig clang 3.7s 3.0s
defconfig gcc 3.5s 3.4s
Whole build, 128-thread Threadripper 9980X, best of N runs:
before after delta
-------------------------------
x86 defconfig, touch mm/vma.c, gcc 9.1s 8.6s -0.43s (-5%)
x86 defconfig, touch mm/vma.c, clang 9.0s 8.0s -1.1s (-12%)
x86 defconfig, clean, gcc 29.5s 28.9s -0.64s (-2%)
x86 defconfig, clean, clang 31.2s 29.8s -1.4s (-5%)
x86 allmodconfig, touch mm/vma.c, gcc 35.8s 35.0s -0.77s (-2%)
x86 allmodconfig, touch mm/vma.c, clang 35.8s 33.7s -2.0s (-6%)
Assisted-by: LLM
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Link: https://patch.msgid.link/20260923-build-speedup-v4-8-73128809a4a4@kernel.org
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
|
||
|
Lorenzo Stoakes (ARM)
|
8050568e0a |
kbuild: only emit vmlinux relocations when required
A kernel build consists of more than one linking pass on vmlinux.o and
vmlinux, at minimum two, and with CONFIG_KALLSYMS and BTF enabled on x86-64
for example there are 5 such stages.
For architectures that build their own relocation tables (x86, riscv, mips,
s390), vmlinux is linked with the --emit-relocs parameter specified.
However, this is only required on the final vmlinux link.
Symbol tables of trial links preceding it don't need it because they
already check that System.map matches kallsyms symbols on each build.
GNU ld is slow at emitting relocation tables, so this results in a
reduction in build time.
Whole build, 128-thread Threadripper 9980X, best of N runs:
before after delta
-------------------------------
x86 defconfig, touch mm/vma.c, gcc 9.5s 9.1s -0.42s (-4%)
x86 defconfig, clean, gcc 30.1s 29.5s -0.59s (-2%)
x86 allmodconfig, touch mm/vma.c, gcc 37.2s 35.8s -1.4s (-4%)
Note that this has little impact on LLVM ld which performs this operation
more efficiently.
Assisted-by: LLM
Reviewed-by: Nicolas Schier <n.schier@fritz.com>
Reviewed-by: Kees Cook <kees@kernel.org>
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Link: https://patch.msgid.link/20260923-build-speedup-v4-6-73128809a4a4@kernel.org
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
|
||
|
Lorenzo Stoakes (ARM)
|
c2970d8b52 |
kbuild: do not sort nm output where the order is irrelevant
Unless instructed otherwise, nm sorts by name.
There are places where this is unnecessary - an invocation from
scripts/sorttable every vmlinux link, scripts/check-function-names.sh run
after every vmlinux.o link, and the x86 VOFFSET and ZOFFSET listings
between vmlinux and bzImage.
Both GNU nm and llvm-nm accept the same '-p' parameter to disable sorting
in these instances, so use that to prevent this unnecessary work.
Each nm run on its own, x86-64, median of 5:
GNU nm 2.47 llvm-nm 22
before after delta before after delta
defconfig
sorttable, nm -S vmlinux 0.080s 0.039s -0.041s 0.265s 0.113s -0.152s
check-function-names.sh 0.072s 0.033s -0.039s 0.244s 0.102s -0.142s
VOFFSET 0.077s 0.034s -0.043s 0.261s 0.113s -0.148s
ZOFFSET 0.007s 0.005s -0.002s 0.005s 0.005s 0.000s
TOTAL -0.125s -0.442s
allmodconfig
sorttable, nm -S vmlinux 0.156s 0.063s -0.093s 0.539s 0.223s -0.316s
check-function-names.sh 0.149s 0.054s -0.095s 0.537s 0.217s -0.320s
VOFFSET 0.146s 0.052s -0.094s 0.524s 0.221s -0.303s
ZOFFSET 0.007s 0.005s -0.002s 0.006s 0.004s -0.002s
TOTAL -0.284s -0.941s
llvm-nm appears to be a lot slower than GNU nm, so these builds naturally
improve the most.
The four run one after another in the serial tail of every build that links
vmlinux so impact kernel builds directly.
In an allmodconfig the decompressor is built in parallel while the modules,
so only the two before the vmlinux link contribute to build time.
The build outputs remain unchanged.
Whole build, 128-thread Threadripper 9980X, best of N runs:
before after delta
---------------------------------
x86 defconfig, touch mm/vma.c, gcc 9.7s 9.5s -0.18s (-2%)
x86 defconfig, touch mm/vma.c, clang 9.3s 9.0s -0.27s (-3%)
x86 defconfig, clean, gcc 30.3s 30.1s -0.16s (-1%)
x86 defconfig, clean, clang 31.7s 31.2s -0.55s (-2%)
x86 allmodconfig, touch mm/vma.c, clang 36.4s 35.8s -0.67s (-2%)
Assisted-by: LLM
Reviewed-by: Nicolas Schier <n.schier@fritz.com>
Reviewed-by: Kees Cook <kees@kernel.org>
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Link: https://patch.msgid.link/20260923-build-speedup-v4-5-73128809a4a4@kernel.org
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
|
||
|
Lorenzo Stoakes (ARM)
|
83da9f3197 |
kallsyms: output binary data to speed output and kallsyms assembly
kallsyms generates an assembly file that consists mostly of .byte entries
containing compressed names, token strings and name-sorted sequence
numbers.
For an x86-64 build with 158k symbols that is a 37 MiB .S file which takes
0.57s to assemble each of the two to three times it is built over a kernel
build.
Each time it is generated it also takes kallsyms a similar amount of time
to output it.
Avoid this overhead by instead outputting this data as binary and importing
it into the assembly using the .incbin directive.
Tables that are wider than a byte remain part of the assembly to ensure
endianness and relative relocations are performed correctly.
With this change, the output assembly file shrinks from 37 MiB to 9.8 MiB,
with a 2.6 MiB binary data file alongside it, and the object remains
identical.
The generated binary file is deleted correctly on build clean along with
all other ephemeral data.
On an x86-64 system with CONFIG_KALLSYMS_ALL set:
before after delta
scripts/kallsyms 0.24s 0.18s 0.06s
assemble 0.57s 0.16s 0.41s
Per kallsyms invocation/assembly, for a total of 0.47s time saving upon
invocation.
An incremental build on the same system was reduced from 11.15s to 9.65s,
indicating a total of 1.5 seconds saved over the build.
The kallsyms runs and their assembly are on the serial tail of every build
that links vmlinux, no-op builds are unchanged.
Whole build, 128-thread Threadripper 9980X, best of N runs:
before after delta
-------------------------------
x86 defconfig, touch mm/vma.c, gcc 10.6s 9.7s -0.89s (-8%)
x86 defconfig, touch mm/vma.c, clang 10.5s 9.3s -1.2s (-11%)
x86 defconfig, clean, gcc 31.3s 30.3s -1.0s (-3%)
x86 defconfig, clean, clang 33.3s 31.7s -1.6s (-5%)
x86 allmodconfig, touch mm/vma.c, gcc 38.7s 37.2s -1.4s (-4%)
x86 allmodconfig, touch mm/vma.c, clang 39.1s 36.4s -2.6s (-7%)
Assisted-by: LLM
Reviewed-by: Kees Cook <kees@kernel.org>
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Link: https://patch.msgid.link/20260923-build-speedup-v4-3-73128809a4a4@kernel.org
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
|
||
|
André Draszik
|
14b8b4bf2d |
kbuild: link-vmlinux.sh: improve detection of third pass requirement
It can happen that symbol sizes within .tmp_vmlinux1.kallsyms.o and
.tmp_vmlinux2.kallsyms.o differ, without affecting file size on disk
because of section alignment and/or padding emitted by the assembler.
link-vmlinux.sh doesn't detect this case currently and keeps using
.tmp_vmlinux2.kallsyms.o to link the final vmlinux. Due to the
different symbol sizes, other symbols are shifted within the final
image compared to .tmp_vmlinux2, and the final comparison of System.map
against "${kallsyms_sysmap}" fails with the message:
Inconsistent kallsyms data
Try "make KALLSYMS_EXTRA_PASS=1" as a workaround
This can happen in particular if the linker emits additional symbols
that might have different names between our (re-)linking steps, e.g.
because those names depend on the virtual address of the symbol. Linker
stubs for ARM errata work-arounds are one such case.
These changed symbol names can cause the output of the token
compression of kallsyms.c to change due to the changed symbol substring
count, which in turn can change the size of the kallsyms_names symbol
itself, causing the potential shift of subsequent symbol addresses in
.tmp_vmlinux2.kallsyms.o and therefore the final image.
Update link-vmlinux.sh to not rely on file size of
.tmp_vmlinux?.kallsyms.o alone but to also consider symbol offsets
within to resolve this, and do a third pass if required.
Signed-off-by: André Draszik <andre.draszik@linaro.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Nicolas Schier <nsc@kernel.org>
Signed-off-by: Nicolas Schier <nsc@kernel.org>
|
||
|
Martin Weiss
|
5a0eefdd32 |
scripts: fix spelling mistakes
Fix two minor typographical issues in two scripts under scripts/. Signed-off-by: Martin Weiss <Martin.weiss2410@gmail.com> Reviewed-by: Nathan Chancellor <nathan@kernel.org> Acked-by: Randy Dunlap <rdunlap@infradead.org> Link: https://patch.msgid.link/20260629082812.551273-1-Martin.weiss2410@gmail.com [nsc: Added minimal commit description] Signed-off-by: Nicolas Schier <nsc@kernel.org> |
||
|
Linus Torvalds
|
41f1a08645 |
Merge tag 'kbuild-7.0-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux
Pull Kbuild/Kconfig updates from Nathan Chancellor:
"Kbuild:
- Drop '*_probe' pattern from modpost section check allowlist, which
hid legitimate warnings (Johan Hovold)
- Disable -Wtype-limits altogether, instead of enabling at W=2
(Vincent Mailhol)
- Improve UAPI testing to skip testing headers that require a libc
when CONFIG_CC_CAN_LINK is not set, opening up testing of headers
with no libc dependencies to more environments (Thomas Weißschuh)
- Update gendwarfksyms documentation with required dependencies
(Jihan LIN)
- Reject invalid LLVM= values to avoid unintentionally falling back
to system toolchain (Thomas Weißschuh)
- Add a script to help run the kernel build process in a container
for consistent environments and testing (Guillaume Tucker)
- Simplify kallsyms by getting rid of the relative base (Ard
Biesheuvel)
- Performance and usability improvements to scripts/make_fit.py
(Simon Glass)
- Minor various clean ups and fixes
Kconfig:
- Move XPM icons to individual files, clearing up GTK deprecation
warnings (Rostislav Krasny)
- Support
depends on FOO if BAR
as syntactic sugar for
depends on FOO || !BAR
(Nicolas Pitre, Graham Roff)
- Refactor merge_config.sh to use awk over shell/sed/grep,
dramatically speeding up processing large number of config
fragments (Anders Roxell, Mikko Rapeli)"
* tag 'kbuild-7.0-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux: (39 commits)
kbuild: remove dependency of run-command on config
scripts/make_fit: Compress dtbs in parallel
scripts/make_fit: Support a few more parallel compressors
kbuild: Support a FIT_EXTRA_ARGS environment variable
scripts/make_fit: Move dtb processing into a function
scripts/make_fit: Support an initial ramdisk
scripts/make_fit: Speed up operation
rust: kconfig: Don't require RUST_IS_AVAILABLE for rustc-option
MAINTAINERS: Add scripts/install.sh into Kbuild entry
modpost: Amend ppc64 save/restfpr symnames for -Os build
MIPS: tools: relocs: Ship a definition of R_MIPS_PC32
streamline_config.pl: remove superfluous exclamation mark
kbuild: dummy-tools: Add python3
scripts: kconfig: merge_config.sh: warn on duplicate input files
scripts: kconfig: merge_config.sh: use awk in checks too
scripts: kconfig: merge_config.sh: refactor from shell/sed/grep to awk
kallsyms: Get rid of kallsyms relative base
mips: Add support for PC32 relocations in vmlinux
Documentation: dev-tools: add container.rst page
scripts: add tool to run containerized builds
...
|
||
|
Ard Biesheuvel
|
a081b57892 |
kallsyms: Get rid of kallsyms relative base
When the kallsyms relative base was introduced, per-CPU variable references on x86_64 SMP were implemented as offsets into the respective per-CPU region, rather than offsets relative to the location of the variable's template in the kernel image, which is how other architectures implement it. This required kallsyms to reason about the difference between the two, and the sign of the value in the kallsyms_offsets[] array was used to distinguish them. This meant that negative offsets were not permitted for ordinary variables, and so it was crucial that the relative base was chosen such that all offsets were positive numbers. This is no longer needed: instead, the offsets can simply be encoded as values in the range -/+ 2 GiB, which is precisely what PC32 relocations provide on most architectures. So it is possible to simplify the logic, and just use _text as the anchor directly, and let the linker calculate the final value based on the location of the entry itself. Some architectures (nios2, extensa) do not support place-relative relocations at all, but these are all 32-bit and non-relocatable, and so there is no need for place-relative relocations in the first place, and the actual symbol values can just be stored directly. This makes all entries in the kallsyms_offsets[] array visible as place-relative references in the ELF metadata, which will be important when implementing ELF-based fg-kaslr. Reviewed-by: Kees Cook <kees@kernel.org> Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Link: https://patch.msgid.link/20260116093359.2442297-6-ardb+git@google.com Signed-off-by: Nathan Chancellor <nathan@kernel.org> |
||
|
Ihor Solodrai
|
26ad5d6e76 |
scripts/gen-btf.sh: Use CONFIG_SHELL for execution
According to the docs [1], kernel build scripts should be executed via
CONFIG_SHELL, which is sh by default.
Fixup gen-btf.sh to be runnable with sh, and use CONFIG_SHELL at every
invocation site.
See relevant discussion for context [2].
[1] https://docs.kernel.org/kbuild/makefiles.html#script-invocation
[2] https://lore.kernel.org/bpf/CAADnVQ+dxmSNoJAGb6xV89ffUCKXe5CJXovXZt22nv5iYFV5mw@mail.gmail.com/
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Tested-by: Gary Guo <gary@garyguo.net>
Reported-by: Gary Guo <gary@garyguo.net>
Suggested-by: Thomas Weißschuh <linux@weissschuh.net>
Fixes:
|
||
|
Ihor Solodrai
|
453dece55b |
scripts/gen-btf.sh: Reduce log verbosity
Remove info messages from gen-btf.sh, as they are unnecessarily detailed and sometimes inaccurate [1]. Verbose log can be produced by passing V=1 to make, which will set -x for the shell. [1] https://lore.kernel.org/bpf/CAADnVQ+biTSDaNtoL=ct9XtBJiXYMUqGYLqu604C3D8N+8YH9A@mail.gmail.com/ Suggested-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev> Link: https://lore.kernel.org/r/20251231183929.65668-1-ihor.solodrai@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org> |
||
|
Ihor Solodrai
|
1a8fa7faf4 |
resolve_btfids: Implement --patch_btfids
Recent changes in BTF generation [1] rely on ${OBJCOPY} command to
update .BTF_ids section data in target ELF files.
This exposed a bug in llvm-objcopy --update-section code path, that
may lead to corruption of a target ELF file. Specifically, because of
the bug st_shndx of some symbols may be (incorrectly) set to 0xffff
(SHN_XINDEX) [2][3].
While there is a pending fix for LLVM, it'll take some time before it
lands (likely in 22.x). And the kernel build must keep working with
older LLVM toolchains in the foreseeable future.
Using GNU objcopy for .BTF_ids update would work, but it would require
changes to LLVM-based build process, likely breaking existing build
environments as discussed in [2].
To work around llvm-objcopy bug, implement --patch_btfids code path in
resolve_btfids as a drop-in replacement for:
${OBJCOPY} --update-section .BTF_ids=${btf_ids} ${elf}
Which works specifically for .BTF_ids section:
${RESOLVE_BTFIDS} --patch_btfids ${btf_ids} ${elf}
This feature in resolve_btfids can be removed at some point in the
future, when llvm-objcopy with a relevant bugfix becomes common.
[1] https://lore.kernel.org/bpf/20251219181321.1283664-1-ihor.solodrai@linux.dev/
[2] https://lore.kernel.org/bpf/20251224005752.201911-1-ihor.solodrai@linux.dev/
[3] https://github.com/llvm/llvm-project/issues/168060#issuecomment-3533552952
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/r/20251231012558.1699758-1-ihor.solodrai@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
|
||
|
Ihor Solodrai
|
522397d05e |
resolve_btfids: Change in-place update with raw binary output
Currently resolve_btfids updates .BTF_ids section of an ELF file
in-place, based on the contents of provided BTF, usually within the
same input file, and optionally a BTF base.
Change resolve_btfids behavior to enable BTF transformations as part
of its main operation. To achieve this, in-place ELF write in
resolve_btfids is replaced with generation of the following binaries:
* ${1}.BTF with .BTF section data
* ${1}.BTF_ids with .BTF_ids section data if it existed in ${1}
* ${1}.BTF.base with .BTF.base section data for out-of-tree modules
The execution of resolve_btfids and consumption of its output is
orchestrated by scripts/gen-btf.sh introduced in this patch.
The motivation for emitting binary data is that it allows simplifying
resolve_btfids implementation by delegating ELF update to the $OBJCOPY
tool [1], which is already widely used across the codebase.
There are two distinct paths for BTF generation and resolve_btfids
application in the kernel build: for vmlinux and for kernel modules.
For the vmlinux binary a .BTF section is added in a roundabout way to
ensure correct linking. The patch doesn't change this approach, only
the implementation is a little different.
Before this patch it worked as follows:
* pahole consumed .tmp_vmlinux1 [2] and added .BTF section with
llvm-objcopy [3] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1
into a .tmp_vmlinux1.bpf.o object [2], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [4],
updating it in-place
After this patch gen-btf.sh implements the following:
* pahole consumes .tmp_vmlinux1 and produces a *detached* file with
raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce
(potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying
BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in
link-vmlinux.sh by objcopy --update-section
For kernel modules, creating a special .bpf.o file is not necessary,
and so embedding of sections data produced by resolve_btfids is
straightforward with objcopy.
With this patch an ELF file becomes effectively read-only within
resolve_btfids, which allows deleting elf_update() call and satellite
code (like compressed_section_fix [5]).
Endianness handling of .BTF_ids data is also changed. Previously the
"flags" part of the section was bswapped in sets_patch() [6], and then
Elf_Type was modified before elf_update() to signal to libelf that
bswap may be necessary. With this patch we explicitly bswap entire
data buffer on load and on dump.
[1] https://lore.kernel.org/bpf/131b4190-9c49-4f79-a99d-c00fac97fa44@linux.dev/
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh?h=v6.18#n110
[3] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c?h=v1.31#n1803
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh?h=v6.18#n284
[5] https://lore.kernel.org/bpf/20200819092342.259004-1-jolsa@kernel.org/
[6] https://lore.kernel.org/bpf/cover.1707223196.git.vmalik@redhat.com/
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Tested-by: Alan Maguire <alan.maguire@oracle.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20251219181825.1289460-3-ihor.solodrai@linux.dev
|
||
|
Linus Torvalds
|
36492b7141 |
Merge tag 'tracepoints-v6.19' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull unused tracepoints update from Steven Rostedt:
"Detect unused tracepoints.
If a tracepoint is defined but never used (TRACE_EVENT() created but
no trace_<tracepoint>() called), it can take up to or more than 5K of
memory each. This can add up as there are around a hundred unused
tracepoints with various configs. That is 500K of wasted memory.
Add a make build parameter of "UT=1" to have the build warn if an
unused tracepoint is detected in the build. This allows detection of
unused tracepoints to be upstream so that outreachy and the mentoring
project can have new developers look for fixing them, without having
these warnings suddenly show up when someone upgrades their kernel.
When all known unused tracepoints are removed, then the "UT=1" build
parameter can be removed and unused tracepoints will always warn. This
will catch new unused tracepoints after the current ones have been
removed.
Summary:
- Separate out elf functions from sorttable.c
Move out the ELF parsing functions from sorttable.c so that the
tracing tooling can use it.
- Add a tracepoint verifier tool to the build process
If "UT=1" is added to the kernel command line, any unused
tracepoints will trigger a warning at build time.
- Do not warn about unused tracepoints for tracepoints that are
exported
There are sever cases where a tracepoint is created by the kernel
and used by modules. Since there's no easy way to detect if these
are truly unused since the users are in modules, if a tracepoint is
exported, assume it will eventually be used by a module. Note,
there's not many exported tracepoints so this should not be a
problem to ignore them.
- Have building of modules also detect unused tracepoints
Do not only check the main vmlinux for unused tracepoints, also
check modules. If a module is defining a tracepoint it should be
using it.
- Add the tracepoint-update program to the ignore file
The new tracepoint-update program needs to be ignored by git"
* tag 'tracepoints-v6.19' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
scripts: add tracepoint-update to the list of ignores files
tracing: Add warnings for unused tracepoints for modules
tracing: Allow tracepoint-update.c to work with modules
tracepoint: Do not warn for unused event that is exported
tracing: Add a tracepoint verification check at build time
sorttable: Move ELF parsing into scripts/elf-parse.[ch]
|
||
|
Steven Rostedt
|
e30f8e61e2 |
tracing: Add a tracepoint verification check at build time
If a tracepoint is defined via DECLARE_TRACE() or TRACE_EVENT() but never called (via the trace_<tracepoint>() function), its metadata is still around in memory and not discarded. When created via TRACE_EVENT() the situation is worse because the TRACE_EVENT() creates metadata that can be around 5k per trace event. Having unused trace events causes several thousand of wasted bytes. Add a verifier that injects a string of the name of the tracepoint it calls that is added to the discarded section "__tracepoint_check". For every builtin tracepoint, its name (which is saved in the in-memory section "__tracepoint_strings") will have its name also in the "__tracepoint_check" section if it is used. Add a new program that is run on build called tracepoint-update. This is executed on the vmlinux.o before the __tracepoint_check section is discarded (the section is discarded before vmlinux is created). This program will create an array of each string in the __tracepoint_check section and then sort it. Then it will walk the strings in the __tracepoint_strings section and do a binary search to check if its name is in the __tracepoint_check section. If it is not, then it is unused and a warning is printed. Note, this currently only handles tracepoints that are builtin and not in modules. Enabling this currently with a given config produces: warning: tracepoint 'sched_move_numa' is unused. warning: tracepoint 'sched_stick_numa' is unused. warning: tracepoint 'sched_swap_numa' is unused. warning: tracepoint 'pelt_hw_tp' is unused. warning: tracepoint 'pelt_irq_tp' is unused. warning: tracepoint 'rcu_preempt_task' is unused. warning: tracepoint 'rcu_unlock_preempted_task' is unused. warning: tracepoint 'xdp_bulk_tx' is unused. warning: tracepoint 'xdp_redirect_map' is unused. warning: tracepoint 'xdp_redirect_map_err' is unused. warning: tracepoint 'vma_mas_szero' is unused. warning: tracepoint 'vma_store' is unused. warning: tracepoint 'hugepage_set_pmd' is unused. warning: tracepoint 'hugepage_set_pud' is unused. warning: tracepoint 'hugepage_update_pmd' is unused. warning: tracepoint 'hugepage_update_pud' is unused. warning: tracepoint 'block_rq_remap' is unused. warning: tracepoint 'xhci_dbc_handle_event' is unused. warning: tracepoint 'xhci_dbc_handle_transfer' is unused. warning: tracepoint 'xhci_dbc_gadget_ep_queue' is unused. warning: tracepoint 'xhci_dbc_alloc_request' is unused. warning: tracepoint 'xhci_dbc_free_request' is unused. warning: tracepoint 'xhci_dbc_queue_request' is unused. warning: tracepoint 'xhci_dbc_giveback_request' is unused. warning: tracepoint 'tcp_ao_wrong_maclen' is unused. warning: tracepoint 'tcp_ao_mismatch' is unused. warning: tracepoint 'tcp_ao_key_not_found' is unused. warning: tracepoint 'tcp_ao_rnext_request' is unused. warning: tracepoint 'tcp_ao_synack_no_key' is unused. warning: tracepoint 'tcp_ao_snd_sne_update' is unused. warning: tracepoint 'tcp_ao_rcv_sne_update' is unused. Some of the above is totally unused but others are not used due to their "trace_" functions being inside configs, in which case, the defined tracepoints should also be inside those same configs. Others are architecture specific but defined in generic code, where they should either be moved to the architecture or be surrounded by #ifdef for the architectures they are for. This tool could be updated to process modules in the future. I'd like to thank Mathieu Desnoyers for suggesting using strings instead of pointers, as using pointers in vmlinux.o required handling relocations and it required implementing almost a full feature linker to do so. To enable this check, run the build with: make UT=1 Note, when all the existing unused tracepoints are removed from the build, the "UT=1" will be removed and this will always be enabled when tracepoints are configured to warn on any new tracepoints. The reason this isn't always enabled now is because it will introduce a lot of warnings for the current unused tracepoints, and all bisects would end at this commit for those warnings. Link: https://lore.kernel.org/all/20250528114549.4d8a5e03@gandalf.local.home/ Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas.schier@linux.dev> Cc: Nick Desaulniers <nick.desaulniers+lkml@gmail.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Randy Dunlap <rdunlap@infradead.org> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Link: https://lore.kernel.org/20251022004452.920728129@kernel.org Suggested-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> # for using strings instead of pointers Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> |
||
|
Josh Poimboeuf
|
f2c356d1d0 |
kbuild,objtool: Defer objtool validation step for CONFIG_KLP_BUILD
In preparation for klp-build, defer objtool validation for CONFIG_KLP_BUILD kernels until the final pre-link archive (e.g., vmlinux.o, module-foo.o) is built. This will simplify the process of generating livepatch modules. Delayed objtool is generally preferred anyway, and is already standard for IBT and LTO. Eventually the per-translation-unit mode will be phased out. Acked-by: Petr Mladek <pmladek@suse.com> Tested-by: Joe Lawrence <joe.lawrence@redhat.com> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org> |
||
|
Alexey Gladkov
|
3328d39a8d |
kbuild: vmlinux.unstripped should always depend on .vmlinux.export.o
Since .vmlinux.export.c is used to add generated by modpost modaliases for builtin modules the .vmlinux.export.o is no longer optional and should always be created. The generation of this file is not dependent on CONFIG_MODULES. Signed-off-by: Alexey Gladkov <legion@kernel.org> Reviewed-by: Nicolas Schier <nsc@kernel.org> Link: https://patch.msgid.link/0e63a9c7741fe8217e4fd7c60afcf057ffa2ef5a.1758182101.git.legion@kernel.org Signed-off-by: Nathan Chancellor <nathan@kernel.org> |
||
|
Linus Torvalds
|
f4d2ef4825 |
Merge tag 'kbuild-v6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild updates from Masahiro Yamada: - Improve performance in gendwarfksyms - Remove deprecated EXTRA_*FLAGS and KBUILD_ENABLE_EXTRA_GCC_CHECKS - Support CONFIG_HEADERS_INSTALL for ARCH=um - Use more relative paths to sources files for better reproducibility - Support the loong64 Debian architecture - Add Kbuild bash completion - Introduce intermediate vmlinux.unstripped for architectures that need static relocations to be stripped from the final vmlinux - Fix versioning in Debian packages for -rc releases - Treat missing MODULE_DESCRIPTION() as an error - Convert Nios2 Makefiles to use the generic rule for built-in DTB - Add debuginfo support to the RPM package * tag 'kbuild-v6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (40 commits) kbuild: rpm-pkg: build a debuginfo RPM kconfig: merge_config: use an empty file as initfile nios2: migrate to the generic rule for built-in DTB rust: kbuild: skip `--remap-path-prefix` for `rustdoc` kbuild: pacman-pkg: hardcode module installation path kbuild: deb-pkg: don't set KBUILD_BUILD_VERSION unconditionally modpost: require a MODULE_DESCRIPTION() kbuild: make all file references relative to source root x86: drop unnecessary prefix map configuration kbuild: deb-pkg: add comment about future removal of KDEB_COMPRESS kbuild: Add a help message for "headers" kbuild: deb-pkg: remove "version" variable in mkdebian kbuild: deb-pkg: fix versioning for -rc releases Documentation/kbuild: Fix indentation in modules.rst example x86: Get rid of Makefile.postlink kbuild: Create intermediate vmlinux build with relocations preserved kbuild: Introduce Kconfig symbol for linking vmlinux with relocations kbuild: link-vmlinux.sh: Make output file name configurable kbuild: do not generate .tmp_vmlinux*.map when CONFIG_VMLINUX_MAP=y Revert "kheaders: Ignore silly-rename files" ... |
||
|
Linus Torvalds
|
dd161f74f8 |
Merge tag 'trace-sorttable-v6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull tracing / sorttable updates from Steven Rostedt: - Implement arm64 build time sorting of the mcount location table When gcc is used to build arm64, the mcount_loc section is all zeros in the vmlinux elf file. The addresses are stored in the Elf_Rela location. To sort at build time, an array is allocated and the addresses are added to it via the content of the mcount_loc section as well as he Elf_Rela data. After sorting, the information is put back into the Elf_Rela which now has the section sorted. - Make sorting of mcount location table for arm64 work with clang as well When clang is used, the mcount_loc section contains the addresses, unlike the gcc build. An array is still created and the sorting works for both methods. - Remove weak functions from the mcount_loc section Have the sorttable code pass in the data of functions defined via 'nm -S' which shows the functions as well as their sizes. Using this information the sorttable code can determine if a function in the mcount_loc section was weak and overridden. If the function is not found, it is set to be zero. On boot, when the mcount_loc section is read and the ftrace table is created, if the address in the mcount_loc is not in the kernel core text then it is removed and not added to the ftrace_filter_functions (the functions that can be attached by ftrace callbacks). - Update and fix the reporting of how much data is used for ftrace functions On boot, a report of how many pages were used by the ftrace table as well as how they were grouped (the table holds a list of sections that are groups of pages that were able to be allocated). The removing of the weak functions required the accounting to be updated. * tag 'trace-sorttable-v6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: scripts/sorttable: Allow matches to functions before function entry scripts/sorttable: Use normal sort if theres no relocs in the mcount section ftrace: Check against is_kernel_text() instead of kaslr_offset() ftrace: Test mcount_loc addr before calling ftrace_call_addr() ftrace: Have ftrace pages output reflect freed pages ftrace: Update the mcount_loc check of skipped entries scripts/sorttable: Zero out weak functions in mcount_loc table scripts/sorttable: Always use an array for the mcount_loc sorting scripts/sorttable: Have mcount rela sort use direct values arm64: scripts/sorttable: Implement sorting mcount_loc at boot for arm64 |
||
|
Ard Biesheuvel
|
e22bbb8e97 |
kbuild: link-vmlinux.sh: Make output file name configurable
In order to introduce an intermediate, non-stripped vmlinux build that can be used by other build steps as an input, pass the output file name to link-vmlinux.sh via its command line. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> |
||
|
Masahiro Yamada
|
ba4d705046 |
kbuild: do not generate .tmp_vmlinux*.map when CONFIG_VMLINUX_MAP=y
Commit
|
||
|
Steven Rostedt
|
ef378c3b82 |
scripts/sorttable: Zero out weak functions in mcount_loc table
When a function is annotated as "weak" and is overridden, the code is not
removed. If it is traced, the fentry/mcount location in the weak function
will be referenced by the "__mcount_loc" section. This will then be added
to the available_filter_functions list. Since only the address of the
functions are listed, to find the name to show, a search of kallsyms is
used.
Since kallsyms will return the function by simply finding the function
that the address is after but before the next function, an address of a
weak function will show up as the function before it. This is because
kallsyms does not save names of weak functions. This has caused issues in
the past, as now the traced weak function will be listed in
available_filter_functions with the name of the function before it.
At best, this will cause the previous function's name to be listed twice.
At worse, if the previous function was marked notrace, it will now show up
as a function that can be traced. Note that it only shows up that it can
be traced but will not be if enabled, which causes confusion.
https://lore.kernel.org/all/20220412094923.0abe90955e5db486b7bca279@kernel.org/
The commit
|
||
|
Brian Gerst
|
01157ddc58 |
kallsyms: Remove KALLSYMS_ABSOLUTE_PERCPU
x86-64 was the only user. Signed-off-by: Brian Gerst <brgerst@gmail.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lore.kernel.org/r/20250123190747.745588-16-brgerst@gmail.com |
||
|
Thomas Weißschuh
|
6e8ba494d8 |
kbuild/btf: Propagate CONFIG_WERROR to resolve_btfids
Use CONFIG_WERROR to also fail on warnings emitted by resolve_btfids. Allow the CI bots to prevent the introduction of new warnings. Signed-off-by: Thomas Weißschuh <linux@weissschuh.net> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20241204-resolve_btfids-v3-2-e6a279a74cfd@weissschuh.net |
||
|
Linus Torvalds
|
6a34dfa15d |
Merge tag 'kbuild-v6.13' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild updates from Masahiro Yamada: - Add generic support for built-in boot DTB files - Enable TAB cycling for dialog buttons in nconfig - Fix issues in streamline_config.pl - Refactor Kconfig - Add support for Clang's AutoFDO (Automatic Feedback-Directed Optimization) - Add support for Clang's Propeller, a profile-guided optimization. - Change the working directory to the external module directory for M= builds - Support building external modules in a separate output directory - Enable objtool for *.mod.o and additional kernel objects - Use lz4 instead of deprecated lz4c - Work around a performance issue with "git describe" - Refactor modpost * tag 'kbuild-v6.13' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (85 commits) kbuild: rename .tmp_vmlinux.kallsyms0.syms to .tmp_vmlinux0.syms gitignore: Don't ignore 'tags' directory kbuild: add dependency from vmlinux to resolve_btfids modpost: replace tdb_hash() with hash_str() kbuild: deb-pkg: add python3:native to build dependency genksyms: reduce indentation in export_symbol() modpost: improve error messages in device_id_check() modpost: rename alias symbol for MODULE_DEVICE_TABLE() modpost: rename variables in handle_moddevtable() modpost: move strstarts() to modpost.h modpost: convert do_usb_table() to a generic handler modpost: convert do_of_table() to a generic handler modpost: convert do_pnp_device_entry() to a generic handler modpost: convert do_pnp_card_entries() to a generic handler modpost: call module_alias_printf() from all do_*_entry() functions modpost: pass (struct module *) to do_*_entry() functions modpost: remove DEF_FIELD_ADDR_VAR() macro modpost: deduplicate MODULE_ALIAS() for all drivers modpost: introduce module_alias_printf() helper modpost: remove unnecessary check in do_acpi_entry() ... |
||
|
Sedat Dilek
|
e6064da646 |
kbuild: rename .tmp_vmlinux.kallsyms0.syms to .tmp_vmlinux0.syms
Change the naming for consistency. While at this, fix the comments in scripts/link-vmlinux.sh. Signed-off-by: Sedat Dilek <sedat.dilek@gmail.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> |
||
|
Masahiro Yamada
|
654102df2a |
kbuild: add generic support for built-in boot DTBs
Some architectures embed boot DTBs in vmlinux. A potential issue for these architectures is a race condition during parallel builds because Kbuild descends into arch/*/boot/dts/ twice. One build thread is initiated by the 'dtbs' target, which is a prerequisite of the 'all' target in the top-level Makefile: ifdef CONFIG_OF_EARLY_FLATTREE all: dtbs endif For architectures that support the built-in boot dtb, arch/*/boot/dts/ is visited also during the ordinary directory traversal in order to build obj-y objects that wrap DTBs. Since these build threads are unaware of each other, they can run simultaneously during parallel builds. This commit introduces a generic build rule to scripts/Makefile.vmlinux to support embedded boot DTBs in a race-free way. Architectures that want to use this rule need to select CONFIG_GENERIC_BUILTIN_DTB. After the migration, Makefiles under arch/*/boot/dts/ will be visited only once to build only *.dtb files. This change also aims to unify the CONFIG options used for built-in DTBs support. Currently, different architectures use different CONFIG options for the same purposes. With this commit, the CONFIG options will be unified as follows: - CONFIG_GENERIC_BUILTIN_DTB This enables the generic rule for built-in boot DTBs. This will be renamed to CONFIG_BUILTIN_DTB after all architectures migrate to the generic rule. - CONFIG_BUILTIN_DTB_NAME This specifies the path to the embedded DTB. (relative to arch/*/boot/dts/) - CONFIG_BUILTIN_DTB_ALL If this is enabled, all DTB files compiled under arch/*/boot/dts/ are embedded into vmlinux. Only used by MIPS. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> |
||
|
Naveen N Rao
|
1198c9c689 |
kbuild: Add generic hook for architectures to use before the final vmlinux link
On powerpc, we would like to be able to make a pass on vmlinux.o and generate a new object file to be linked into vmlinux. Add a generic pass in Makefile.vmlinux that architectures can use for this purpose. Architectures need to select CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX and must provide arch/<arch>/tools/Makefile with .arch.vmlinux.o target, which will be invoked prior to the final vmlinux link step. Acked-by: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Naveen N Rao <naveen@kernel.org> Signed-off-by: Hari Bathini <hbathini@linux.ibm.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://patch.msgid.link/20241030070850.1361304-12-hbathini@linux.ibm.com |
||
|
Linus Torvalds
|
68e5c7d4ce |
Merge tag 'kbuild-v6.12' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild updates from Masahiro Yamada: - Support cross-compiling linux-headers Debian package and kernel-devel RPM package - Add support for the linux-debug Pacman package - Improve module rebuilding speed by factoring out the common code to scripts/module-common.c - Separate device tree build rules into scripts/Makefile.dtbs - Add a new script to generate modules.builtin.ranges, which is useful for tracing tools to find symbols in built-in modules - Refactor Kconfig and misc tools - Update Kbuild and Kconfig documentation * tag 'kbuild-v6.12' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (51 commits) kbuild: doc: replace "gcc" in external module description kbuild: doc: describe the -C option precisely for external module builds kbuild: doc: remove the description about shipped files kbuild: doc: drop section numbering, use references in modules.rst kbuild: doc: throw out the local table of contents in modules.rst kbuild: doc: remove outdated description of the limitation on -I usage kbuild: doc: remove description about grepping CONFIG options kbuild: doc: update the description about Kbuild/Makefile split kbuild: remove unnecessary export of RUST_LIB_SRC kbuild: remove append operation on cmd_ld_ko_o kconfig: cache expression values kconfig: use hash table to reuse expressions kconfig: refactor expr_eliminate_dups() kconfig: add comments to expression transformations kconfig: change some expr_*() functions to bool scripts: move hash function from scripts/kconfig/ to scripts/include/ kallsyms: change overflow variable to bool type kallsyms: squash output_address() kbuild: add install target for modules.builtin.ranges scripts: add verifier script for builtin module range data ... |
||
|
Linus Torvalds
|
440b652328 |
Merge tag 'bpf-next-6.12' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
Pull bpf updates from Alexei Starovoitov:
- Introduce '__attribute__((bpf_fastcall))' for helpers and kfuncs with
corresponding support in LLVM.
It is similar to existing 'no_caller_saved_registers' attribute in
GCC/LLVM with a provision for backward compatibility. It allows
compilers generate more efficient BPF code assuming the verifier or
JITs will inline or partially inline a helper/kfunc with such
attribute. bpf_cast_to_kern_ctx, bpf_rdonly_cast,
bpf_get_smp_processor_id are the first set of such helpers.
- Harden and extend ELF build ID parsing logic.
When called from sleepable context the relevants parts of ELF file
will be read to find and fetch .note.gnu.build-id information. Also
harden the logic to avoid TOCTOU, overflow, out-of-bounds problems.
- Improvements and fixes for sched-ext:
- Allow passing BPF iterators as kfunc arguments
- Make the pointer returned from iter_next method trusted
- Fix x86 JIT convergence issue due to growing/shrinking conditional
jumps in variable length encoding
- BPF_LSM related:
- Introduce few VFS kfuncs and consolidate them in
fs/bpf_fs_kfuncs.c
- Enforce correct range of return values from certain LSM hooks
- Disallow attaching to other LSM hooks
- Prerequisite work for upcoming Qdisc in BPF:
- Allow kptrs in program provided structs
- Support for gen_epilogue in verifier_ops
- Important fixes:
- Fix uprobe multi pid filter check
- Fix bpf_strtol and bpf_strtoul helpers
- Track equal scalars history on per-instruction level
- Fix tailcall hierarchy on x86 and arm64
- Fix signed division overflow to prevent INT_MIN/-1 trap on x86
- Fix get kernel stack in BPF progs attached to tracepoint:syscall
- Selftests:
- Add uprobe bench/stress tool
- Generate file dependencies to drastically improve re-build time
- Match JIT-ed and BPF asm with __xlated/__jited keywords
- Convert older tests to test_progs framework
- Add support for RISC-V
- Few fixes when BPF programs are compiled with GCC-BPF backend
(support for GCC-BPF in BPF CI is ongoing in parallel)
- Add traffic monitor
- Enable cross compile and musl libc
* tag 'bpf-next-6.12' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (260 commits)
btf: require pahole 1.21+ for DEBUG_INFO_BTF with default DWARF version
btf: move pahole check in scripts/link-vmlinux.sh to lib/Kconfig.debug
btf: remove redundant CONFIG_BPF test in scripts/link-vmlinux.sh
bpf: Call the missed kfree() when there is no special field in btf
bpf: Call the missed btf_record_free() when map creation fails
selftests/bpf: Add a test case to write mtu result into .rodata
selftests/bpf: Add a test case to write strtol result into .rodata
selftests/bpf: Rename ARG_PTR_TO_LONG test description
selftests/bpf: Fix ARG_PTR_TO_LONG {half-,}uninitialized test
bpf: Zero former ARG_PTR_TO_{LONG,INT} args in case of error
bpf: Improve check_raw_mode_ok test for MEM_UNINIT-tagged types
bpf: Fix helper writes to read-only maps
bpf: Remove truncation test in bpf_strtol and bpf_strtoul helpers
bpf: Fix bpf_strtol and bpf_strtoul helpers for 32bit
selftests/bpf: Add tests for sdiv/smod overflow cases
bpf: Fix a sdiv overflow issue
libbpf: Add bpf_object__token_fd accessor
docs/bpf: Add missing BPF program types to docs
docs/bpf: Add constant values for linkages
bpf: Use fake pt_regs when doing bpf syscall tracepoint tracing
...
|
||
|
Masahiro Yamada
|
42450f7a90 |
btf: move pahole check in scripts/link-vmlinux.sh to lib/Kconfig.debug
When DEBUG_INFO_DWARF5 is selected, pahole 1.21+ is required to enable
DEBUG_INFO_BTF.
When DEBUG_INFO_DWARF4 or DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT is selected,
DEBUG_INFO_BTF can be enabled without pahole installed, but a build error
will occur in scripts/link-vmlinux.sh:
LD .tmp_vmlinux1
BTF: .tmp_vmlinux1: pahole (pahole) is not available
Failed to generate BTF for vmlinux
Try to disable CONFIG_DEBUG_INFO_BTF
We did not guard DEBUG_INFO_BTF by PAHOLE_VERSION when previously
discussed [1].
However, commit
|
||
|
Masahiro Yamada
|
c980dc9c67 |
btf: remove redundant CONFIG_BPF test in scripts/link-vmlinux.sh
CONFIG_DEBUG_INFO_BTF depends on CONFIG_BPF_SYSCALL, which in turn selects CONFIG_BPF. When CONFIG_DEBUG_INFO_BTF=y, CONFIG_BPF=y is always met. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Alan Maguire <alan.maguire@oracle.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Reviewed-by: Nathan Chancellor <nathan@kernel.org> Link: https://lore.kernel.org/r/20240913173759.1316390-1-masahiroy@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org> |
||
|
Tony Battersby
|
dde60e7d10 |
kbuild: remove recent dependency on "truncate" program
Remove the recently-added dependency on the truncate program for
building the kernel. truncate is not available when building the kernel
under Yocto. It could be added, but it would be better just to avoid
the unnecessary dependency.
Fixes:
|
||
|
Linus Torvalds
|
3f44ae972a |
Merge tag 'kbuild-fixes-v6.11-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild fixes from Masahiro Yamada: - Eliminate the fdtoverlay command duplication in scripts/Makefile.lib - Fix 'make compile_commands.json' for external modules - Ensure scripts/kconfig/merge_config.sh handles missing newlines - Fix some build errors on macOS * tag 'kbuild-fixes-v6.11-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: kbuild: fix typos "prequisites" to "prerequisites" Documentation/llvm: turn make command for ccache into code block kbuild: avoid scripts/kallsyms parsing /dev/null treewide: remove unnecessary <linux/version.h> inclusion scripts: kconfig: merge_config: config files: add a trailing newline Makefile: add $(srctree) to dependency of compile_commands.json target kbuild: clean up code duplication in cmd_fdtoverlay |
||
|
Song Liu
|
020925ce92 |
kallsyms: Do not cleanup .llvm.<hash> suffix before sorting symbols
Cleaning up the symbols causes various issues afterwards. Let's sort
the list based on original name.
Signed-off-by: Song Liu <song@kernel.org>
Fixes:
|
||
|
Masahiro Yamada
|
1472464c62 |
kbuild: avoid scripts/kallsyms parsing /dev/null
On macOS, as reported by Daniel Gomez, getline() sets ENOTTY to errno
if it is requested to read from /dev/null.
If this is worth fixing, I would rather pass an empty file to
scripts/kallsyms instead of adding the ugly #ifdef __APPLE__.
Fixes:
|
||
|
Jann Horn
|
64e166099b |
kallsyms: get rid of code for absolute kallsyms
Commit
|
||
|
Masahiro Yamada
|
b1a9a5e047 |
kbuild: merge temporary vmlinux for BTF and kallsyms
CONFIG_DEBUG_INFO_BTF=y requires one additional link step.
(.tmp_vmlinux.btf)
CONFIG_KALLSYMS=y requires two additional link steps.
(.tmp_vmlinux.kallsyms1 and .tmp_vmlinux.kallsyms2)
Enabling both requires three additional link steps.
When CONFIG_DEBUG_INFO_BTF=y and CONFIG_KALLSYMS=y, the current build
process is as follows:
KSYMS .tmp_vmlinux.kallsyms0.S
AS .tmp_vmlinux.kallsyms0.o
LD .tmp_vmlinux.btf # temporary vmlinux for BTF
BTF .btf.vmlinux.bin.o
LD .tmp_vmlinux.kallsyms1 # temporary vmlinux for kallsyms step 1
NM .tmp_vmlinux.kallsyms1.syms
KSYMS .tmp_vmlinux.kallsyms1.S
AS .tmp_vmlinux.kallsyms1.o
LD .tmp_vmlinux.kallsyms2 # temporary vmlinux for kallsyms step 2
NM .tmp_vmlinux.kallsyms2.syms
KSYMS .tmp_vmlinux.kallsyms2.S
AS .tmp_vmlinux.kallsyms2.o
LD vmlinux # final vmlinux
This is redundant because the BTF generation and the kallsyms step 1 can
be performed against the same temporary vmlinux.
When both CONFIG_DEBUG_INFO_BTF and CONFIG_KALLSYMS are enabled, we can
reduce the number of link steps by one.
This commit changes the build process as follows:
KSYMS .tmp_vmlinux0.kallsyms.S
AS .tmp_vmlinux0.kallsyms.o
LD .tmp_vmlinux1 # temporary vmlinux for BTF and kallsyms step 1
BTF .tmp_vmlinux1.btf.o
NM .tmp_vmlinux1.syms
KSYMS .tmp_vmlinux1.kallsyms.S
AS .tmp_vmlinux1.kallsyms.o
LD .tmp_vmlinux2 # temporary vmlinux for kallsyms step 2
NM .tmp_vmlinux2.syms
KSYMS .tmp_vmlinux2.kallsyms.S
AS .tmp_vmlinux2.kallsyms.o
LD vmlinux # final vmlinux
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
|
||
|
Masahiro Yamada
|
c442db3f49 |
kbuild: remove PROVIDE() for kallsyms symbols
This reimplements commit
|
||
|
Masahiro Yamada
|
ddf4132983 |
kbuild: refactor variables in scripts/link-vmlinux.sh
Clean up the variables in scripts/link-vmlinux.sh - Specify the extra objects directly in vmlinux_link() - Move the AS rule to kallsyms() - Set kallsymso and btf_vmlinux_bin_o where they are generated - Remove unneeded variable, kallsymso_prev - Introduce the btf_data variable - Introduce the strip_debug flag instead of checking the output name No functional change intended. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu> |
||
|
Richard Acayan
|
96c965667b |
kbuild: explicitly run mksysmap as sed script from link-vmlinux.sh
In commit |
||
|
Masahiro Yamada
|
3c562a70cf |
kbuild: remove a stale comment about cleaning in link-vmlinux.sh
Remove the left-over of commit
|
||
|
Masahiro Yamada
|
3430f65d61 |
kbuild: fix short log for AS in link-vmlinux.sh
In convention, short logs print the output file, not the input file. Let's change the suffix for 'AS' since it assembles *.S into *.o. [Before] LD .tmp_vmlinux.kallsyms1 NM .tmp_vmlinux.kallsyms1.syms KSYMS .tmp_vmlinux.kallsyms1.S AS .tmp_vmlinux.kallsyms1.S LD .tmp_vmlinux.kallsyms2 NM .tmp_vmlinux.kallsyms2.syms KSYMS .tmp_vmlinux.kallsyms2.S AS .tmp_vmlinux.kallsyms2.S LD vmlinux [After] LD .tmp_vmlinux.kallsyms1 NM .tmp_vmlinux.kallsyms1.syms KSYMS .tmp_vmlinux.kallsyms1.S AS .tmp_vmlinux.kallsyms1.o LD .tmp_vmlinux.kallsyms2 NM .tmp_vmlinux.kallsyms2.syms KSYMS .tmp_vmlinux.kallsyms2.S AS .tmp_vmlinux.kallsyms2.o LD vmlinux Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> |
||
|
Masahiro Yamada
|
b18b047002 |
kbuild: change scripts/mksysmap into sed script
The previous commit removed the subshell execution from scripts/mksysmap, which is now simple enough to become a sed script. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> |
||
|
Masahiro Yamada
|
04b8cb0945 |
kbuild: avoid unneeded kallsyms step 3
Since commit |
||
|
Nathan Chancellor
|
e3a9ee963a |
kbuild: Fix changing ELF file type for output of gen_btf for big endian
Commit |
||
|
Linus Torvalds
|
d55571c008 |
Merge tag 'kbuild-v6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild updates from Masahiro Yamada: - Refactor scripts/kallsyms to make it faster and easier to maintain - Clean up menuconfig - Provide Clang with hard-coded target triple instead of CROSS_COMPILE - Use -z pack-relative-relocs flags instead of --use-android-relr-tags for arm64 CONFIG_RELR - Add srcdeb-pkg target to build only a Debian source package - Add KDEB_SOURCE_COMPRESS option to specify the compression for a Debian source package - Misc cleanups and fixes * tag 'kbuild-v6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: kbuild: deb-pkg: specify targets in debian/rules as .PHONY sparc: unify sparc32/sparc64 archhelp kbuild: rpm-pkg: remove kernel-drm PROVIDES kbuild: deb-pkg: add KDEB_SOURCE_COMPRESS to specify source compression kbuild: add srcdeb-pkg target Makefile: use -z pack-relative-relocs kbuild: clang: do not use CROSS_COMPILE for target triple kconfig: menuconfig: reorder functions to remove forward declarations kconfig: menuconfig: remove unused M_EVENT macro kconfig: menuconfig: remove OLD_NCURSES macro kbuild: builddeb: Eliminate debian/arch use scripts/kallsyms: update the usage in the comment block scripts/kallsyms: decrease expand_symbol() / cleanup_symbol_name() calls scripts/kallsyms: change the output order scripts/kallsyms: move compiler-generated symbol patterns to mksysmap scripts/kallsyms: exclude symbols generated by itself dynamically scripts/mksysmap: use sed with in-line comments scripts/mksysmap: remove comments described in nm(1) scripts/kallsyms: remove redundant code for omitting U and N kallsyms: expand symbol name into comment for debugging |
||
|
Masahiro Yamada
|
ca09bf48f9 |
scripts/kallsyms: exclude symbols generated by itself dynamically
Drop the symbols generated by scripts/kallsyms itself automatically instead of maintaining the symbol list manually. Pass the kallsyms object from the previous kallsyms step (if it exists) as the third parameter of scripts/mksysmap, which will weed out the generated symbols from the input to the next kallsyms step. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> |
||
|
Andrew Morton
|
4b3d049f1c |
scripts/link-vmlinux.sh: fix error message presentation
This comes out as Try make KALLSYMS_EXTRA_PASS=1 as a workaround but we want quotes: Try "make KALLSYMS_EXTRA_PASS=1" as a workaround Link: https://lkml.kernel.org/r/202303042034.Cjc7JTd0-lkp@intel.com Cc: kernel test robot <lkp@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |