mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
synced 2026-10-26 10:57:38 +08:00
master
2898
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Linus Torvalds
|
56070f00d0 |
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
Pull KVM updates from Paolo Bonzini:
"ARM:
- pKVM state isolation: years after pKVM started being enabled
upstream, we finally get the basics in the form of state isolation.
The vcpu state is now off-limit for the host for protected VMs
- pKVM also grows a hypervisor-private memory allocator, allowing it
to deal with sub-page allocation, much like everywhere else
- pKVM also lands a trace_printk() equivalent, which hopefully will
make debugging a bit easier
- First shot at annotating portions of the KVM/arm64 code to be
extracted and built on s390. The real fun is yet to come
- Allow pre-faulting of stage-2 canonical mappings from userspace,
joining x86 and s390 in implementing this feature
- Add a reverse mapping infrastructure for shadow stage-2,
significantly improving performance when MMU notifiers kick in, and
avoiding full unmap of all shadow tables for this VM
- Next step of GICv5 enablement, adding support for the IRS block,
and providing both SPI and LPI support. This is the first point
where KVM/arm64 can support an SMP GICv5 guest
- As a preliminary to eventually enabling CCA, add a bunch of
infrastructure to KVM/arm64, splitting some of the basic operations
into individual backends (normal VM, normal pKVM VM, protected pKVM
VM). This allows CCA to later slot in without polluting the rest of
KVM with its firmware-driven nonsense
- Make ID registers immutable from the point where a vcpu is able to
run, even if it fails to run. This prevents KVM from sampling
untrusted values
- Restore some type checking to the EL2 object. KVM/arm64 originally
had some strong type checking between the kernel and the EL2
object, but lost it along the way when the private hypercall
interface was (somewhat pointlessly) converted to SMCCC. Type
checking is back, syscall style, and with added sparse goodies
- Expand the FFA support to FFA_NOTIFICATION_*, and add some
sanitisation of the protocol that pKVM relays to the "trusted"
runtime
- Make GICv3 RD assignment more robust to userspace nonsense
- Convert the remaining MMU fault handler backends to
kvm_s2_fault_vma_info and co., fixing a number of bugs in the
process
- Fix the SError/SEA selftests to be both more robust and less
braindead
- Fix a smattering of documentation bugs, some of them as a result of
our AI friends spitting out credible assertions based on
non-existing symbols
- The usual collection of low impact bug fixes (SMCCC, timers, PMU)
Loongarch:
- Add separate vmid support
- Simplify MSI interrupt injection
- Add crash hypercall support
- Allow to set pv_feature until vCPU run
- Enable HW page table walk automatically in selftests
s390:
- rework page allocation for secure guest variable storage
- Selftests and documentation improvements
- Replace get_zeroed_page with kzalloc
- Fix corner cases of floating interrupt injection
- Initial arm64 on s390 support
x86:
- Clean up the handling of vcpu->arch.xen.long_mode to be consistent
and correctly handle 32-bit/64-bit alignment, and various other bug
fixes have accumulated over the months since v1
- Grab KVM's SRCU "lock" in kvm_zap_gfn_range() to fix a recurring
bug where callers fail to do so
- Fix a variety of issues in the pre-fault code that are mostly
benign in the current code base, but will show up in big ways
if/when the pre-faulting logic is used to implementing
prefetch-on-fault for the TDP MMU
- For PREEMPT_RT=y kernels, drop the rmaps' individual bit-spinlock
and just walk the rmaps' with mmu_lock taken. On the write side
(which always uses mmu_lock), the bit-spinlock was assuming that
preemption was disabled, but that is *not* true when rwlocks become
sleepable
- Prepare a memory_fault exit (paired with -EFAULT) when KVM hits an
error PFN, e.g. so that userspace can resolve transient issues
instead of killing the guest. Convert a variety of WARN()+EFAULT
paths into KVM_BUG_ON()+EIO so that KVM can hard commit to filling
memory_fault on -EFAULTs that occur in the page fault path
- Fix a NX hugepage un-accounting bug where KVM would always uncharge
the shadow MMU, even when handling TDP MMU pages
- Mask off the address space ID when checking for illegal usage of
kvm_set_internal_memslot() to prevent false negatives on the sanity
check
- Explicitly zero reserved flags when delivering a SynIC message to
avoid leaking a byte of stack information to the guest
- Rework TDX's handling of the canonical MMU root level to not rely
on userspace to fill CPUID correctly, as doing so lets userspace
trivially trigger a KVM_BUG_ON()
- Reject KVM_SET_NESTED_STATE if userspace "restores" L1 with
EFER.LMA=1 && EFER.LME=0, which confuses KVM into trying to
configure an 64-bit MMU without PAE
- Ignore EFER.LMA (clear it from KVM's cached save state) if
EFER.LME=0 when preparing L2 state, to fix another way to confuse
KVM's MMU (and because real hardware also ignores EFER.LMA)
- Improve KVM's sanity checks for a misconfigured MMU to (hopefully)
detect all of the ways KVM can botch EFER.LMA vs. CR4.PAE
- Remove the "full" out-param from kvm_x86_ops.flush_tlb_gva() as the
param is only used to avoid what effectively amounts to a single
memory write
- Fix a mostly-benign bug where KVM can write through a guest's
read-only mapping due to not checking the error code when
expediting emulated MMIO page faults
- Honor the guest's view of the "EFER.LMSLE must be zero" defeature
so that userspace can provide migration compatibility between Rome
and Milan+
- Pend a KVM_REQ_EVENT request when an exception is morphed into a
nested VM-Exit, otherwise KVM can fail to actually deliver the
nested VM-Exit, e.g. when injecting a page fault
- Add support for PML on SVM
- Add support for virtualizing Bus Lock Detect on SVM, and fix a
number of DR6 bugs and warts along the way
- Fix a variety of warts in KVM's handling of guest TSC, kvmclock,
and timekeeping in general, as a (big) first step towards
addressing several longstanding problems with the kvmclock
implementation. Most notably, fix a precision issue where
get_kvmclock_ns() computed the clock from the *host* TSC without
applying guest TSC scaling, causing systemic drift from the values
the guest computes from its own TSC
- Rip out KVM's shadow timekeeping structures, now that ktime
snapshots are powerful enough to replace it
- Misc fixes and cleanups
x86 (AMD):
- Fix a benign-in-practice bug where KVM unintentionally sets a
non-zero error code when delivering exceptions to SEV-ES+ guests
- Force a new ASID in both VMCBs on pCPU migration to fix a bug where
KVM can run a VMCB with a stale ASID due to the tracking being
per-CPU, not per-VMCB
- Return INVALID_INPUT to the guest instead of exiting to userspace
with -EIO if KVM can't access "guest request" buffer, i.e. try to
keep the guest alive if the guest or userspace goofs
x86 (Intel):
- Enable Notify VM-Exits (now called Instruction Timeout in the SDM)
and Bus Lock VM-Exits for TDX guests when they're enabled by
userspace (KVM forgot to wire up the TDX Module knobs)
- Move all common (between VMX and TDX) APIs/helpers to
common.h/main.c, and change all namespace prefixes from "vmx" to
"vt"
- Fix a thankfully benign bug (due to other areas of KVM being
hardened) where userspace can bypass INIT_MEM_REGION's address wrap
check by providing a nr_pages that will overflow a u64 when
converted to bytes, and fix an also- benign off-by-one bug in the
check (inclusive vs. exclusive)
- Set the vmcs.GUEST_ACTIVITY_STATE back to ACTIVE when userspace
sets KVM's mp_state to anything other than HALTED, e.g. so that a
vCPU doesn't get stuck in HLT if userspace emulates RESET
guest_memfd:
- Take the invalidate lock when unbinding a dying file to fix a race
with kvm_gmem_invalidate_end() that results in KVM leaving the MMU
invalidation count elevated. Not taking the lock in the first place
was pure misguided paranoid
- When binding a memslot, purge the bindings if storing the new
memslot fails, as Xarray's "store range" API can subtly fail
partway through
- Establish memslot<=>guest_memfd bindings *after* the memslot is
ready, as the memslot becomes reachable the instant the binding is
installed
- Don't grab a page refcount when reloading an SNP VMSA and stop
returning a "struct page" from guest_memfd's PFN lookup altogether,
to ensure that KVM doesn't hold any unnecessary refcounts (the
lifetime of the page is ensured by MMU notifiers and similar
hooks). Preventing unnecessary refcounts greatly simplifies
supporting in-place SHARED=>PRIVATE conversions
- Add support for in-place PRIVATE<=>SHARED conversion, i.e. allow a
single guest_memfd instance to be used for both PRIVATE and SHARED
variations of CoCo guests' memory (though not at the same time).
This eliminates the need to double allocate guest memory for TDX
and SNP, and is basically a hard requirement for pKVM, which relies
on being able to preserve memory contents on conversions
- Hack-a-fix a deadlock bug in ->error_remove_folio() by returning
-EBUSY if the filemap invalidate lock is contended. Punt on a
proper fix for the moment, as a proper fix likely involves
reworking truncate_error_folio()
- Add a pile of selftests for in-place conversions
- Misc guest_memfd cleanups
Generic:
- Move some configuration items to preprocessor symbols, to help with
arm-on-s390's need to compile the same code twice with different
values
- Allow KVM implementations to choose alternative device names, also
for s390's support of ARM virtualmachines
- Switch the KVM-VFIO interface and external consumers over to
standard file-based reference counting, eliminating all external
KVM symbol exports
Selftests:
- Rework KVM selftest's handling of memory regions all so that PPC
can force its page table allocations to be naturally aligned
- PPC support
- Fix emulation of VMPTRST and VMPTRLD when eVMCS is in-use to match
real Hyper-V's behavior
- Saturate L2's TSC frequency if stacking L1 and L2 scaling
multipliers exceeds what hardware supports, as attempting to run L2
with an illegal multiplier will either cause VM-Entry to fail (VMX)
or trigger a #GP when writing MSR_AMD64_TSC_RATIO (SVM). The #GP is
thankfully benign in the grand scheme as even "unsafe" WRMSRs have
exception fixup
- Rework KVM selftest's handling of memory regions so that PPC can
force its page table allocations to be naturally aligned
- Clean up selftest's VMX instruction helpers, and drop an absurd
number of completely useless return values, e.g. functions that
unconditionally return %true and then have every caller assert
success
- Add selftests support for Zhaoxin CPUs
- Overhaul the TSC scaling test to add a regression test for the
saturation behavior, and to increase its coverage in general
- Fix selftests usage various NUMA syscalls to account for an
undocumented, but known since 2007, off-by-one kernel bug. Add
helpers to try and avoid re-introducing the same flaws in the
future
- Automatically run xAPIC IPI migration test (between NUMA nodes)
when possible, as the entire point of the test is verify KVM's
handling of page migration of the APIC backing page
- Misc cleanups"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (506 commits)
KVM: always make in-place conversion a compile-time constant when possible
KVM: x86: leave the legacy private attribute in for TDX and SEV
KVM: split CONFIG_KVM_PRIVATE_MEMATTR_VM out of KVM_VM_MEMORY_ATTRIBUTES
selftests: kvm: remove useless comma operator
arm64: Use proper include variant
MAINTAINERS: Add entry for ARM on s390 KVM
KVM: s390: arm64: Integrate arm on s390
KVM: s390: arm64: Implement basic page fault handler
KVM: s390: arm64: Implement vCPU IOCTLs
KVM: s390: arm64: Implement vm/vcpu create destroy
KVM: s390: arm64: Implement required functions
KVM: s390: Add basic arm64 kvm module
s390/hwcaps: Report SAE support as hwcap
KVM: s390: arm64: Introduce host definitions
s390: Introduce Start Arm Execution instruction
KVM: s390: Use arm64 code
s390/tools: Use arm64 headers
KVM: s390: Use dedicated function for migration mode
s390/Kconfig: remove PCI dependency from HAS_IOMEM
KVM: s390: Kick PV cpus at the right time for service irqs
...
|
||
|
Paolo Bonzini
|
164405e8e9 |
KVM: always make in-place conversion a compile-time constant when possible
Remove the unnecessary static call indirection and make gmem_in_place_conversion a constant if CONFIG_KVM_PRIVATE_MEMATTR_VM is false. Require that CONFIG_KVM_PRIVATE_MEMATTR_VM be enabled together with CONFIG_KVM_GUEST_MEMFD and with a definition of kvm_arch_has_private_mem. This way, the three possible cases can be expressed without nested #ifs. Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> |
||
|
Paolo Bonzini
|
28d5df7b72 |
KVM: split CONFIG_KVM_PRIVATE_MEMATTR_VM out of KVM_VM_MEMORY_ATTRIBUTES
With Linux 7.4 the "private" memory attribute was moved from the VM to the individual guest_memfd file descriptor. The old semantics are available for backwards compatibility via CONFIG_KVM_VM_MEMORY_ATTRIBUTES. However, this ties support for VM memory attributes entirely to a backwards compatibility symbol, which is a problem when adding VM memory attributes other than PRIVATE. Thus, make CONFIG_KVM_VM_MEMORY_ATTRIBUTES the equivalent of the old CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES, and use a more descriptive name for the private memory attribute. Do this now, instead of waiting for more attributes to come, before distros start adjusting their build configuration. Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> |
||
|
Linus Torvalds
|
00b04f4495 |
Merge tag 'vfs-7.4-rc1.idmap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs idmap constification from Christian Brauner:
"This makes struct mnt_idmap const throughout the tree.
A mount's idmapping is immutable. The only thing modified after
alloc_mnt_idmap() is the reference count and that is hidden behind
mnt_idmap_get() and mnt_idmap_put(). Everything else only reads from
it. That is the same model struct cred uses and the idmapping is
sensitive enough to deserve it.
The conversion goes bottom up through the vfs{g,u}id and fs{g,u}id
helpers, the privilege checks, acl, xattr and quota and then every
inode operation that takes an idmap, so ->permission(), ->getattr(),
->setattr(), ->create(), ->mkdir(), ->mknod(), ->symlink(),
->rename(), ->tmpfile(), ->get_acl(), ->set_acl() and ->fileattr_set()
with their implementations in every filesystem, plus the LSM hooks.
struct vfsmount, struct mount_kattr and struct kstatmount store a
const pointer and nop_mnt_idmap and invalid_mnt_idmap live in rodata,
so anything that tries to modify them faults instead of silently
corrupting the initial idmapping"
* tag 'vfs-7.4-rc1.idmap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (27 commits)
fs: make nop_mnt_idmap and invalid_mnt_idmap const
fs: port mnt_idmap() and file_mnt_idmap() to const mnt_idmap
fs: port vfs_*() helpers to const mnt_idmap
fs: port ->setattr() to pass const mnt_idmap
fs: port ->getattr() to pass const mnt_idmap
fs: port ->create() to pass const mnt_idmap
fs: port ->symlink() to pass const mnt_idmap
fs: port ->mkdir() to pass const mnt_idmap
fs: port ->rename() to pass const mnt_idmap
fs: port ->mknod() to pass const mnt_idmap
fs: port ->tmpfile() to pass const mnt_idmap
fs: port ->get_acl() to pass const mnt_idmap
fs: port ->set_acl() to pass const mnt_idmap
fs: port ->fileattr_set() to pass const mnt_idmap
fs: port xattr to const mnt_idmap
fs: port ->permission() to pass const mnt_idmap
fs: port acl to const mnt_idmap
fs: port inode_init_owner() to const mnt_idmap
fs: port inode_owner_or_capable() to const mnt_idmap
fs: port privilege checking helpers to const mnt_idmap
...
|
||
|
Paolo Bonzini
|
7213f4e515 |
Merge tag 'kvm-x86-misc-7.4' of https://github.com/kvm-x86/linux into HEAD
KVM x86 misc changes for 7.4 - Mask off the address space ID when checking for illegal usage of kvm_set_internal_memslot() to prevent false negatives on the sanity check. - Explicitly zero reserved flags when delivering a SynIC message to avoid leaking a byte of stack information to the guest. - Rework TDX's handling of the canonical MMU root level to not rely on userspace to fill CPUID correctly, as doing so lets userspace trivially trigger a KVM_BUG_ON(). - Reject KVM_SET_NESTED_STATE if userspace "restores" L1 with EFER.LMA=1 && EFER.LME=0, which confuses KVM into trying to configure an 64-bit MMU without PAE. - Ignore EFER.LMA (clear it from KVM's cached save state) if EFER.LME=0 when preparing L2 state, to fix another way to confuse KVM's MMU (and because real hardware also ignores EFER.LMA). - Improve KVM's sanity checks for a misconfigured MMU to (hopefully) detect all of the ways KVM can botch EFER.LMA vs. CR4.PAE. - Remove the "full" out-param from kvm_x86_ops.flush_tlb_gva() as the param is only used to avoid what effectively amounts to a single memory write. - Fix a mostly-benign bug where KVM can write through a guest's read-only mapping due to not checking the error code when expediting emulated MMIO page faults. - Honor the guest's view of the "EFER.LMSLE must be zero" defeature so that userspace can provide migration compatibility between Rome and Milan+. - Pend a KVM_REQ_EVENT request when an exception is morphed into a nested VM-Exit, otherwise KVM can fail to actually deliver the nested VM-Exit, e.g. when injecting a page fault. - Add support for PML on SVM. - Add support for virtualizing Bus Lock Detect on SVM, and fix a number of DR6 bugs and warts along the way. - Misc fixes and cleanups. |
||
|
Paolo Bonzini
|
29c925376d |
Merge tag 'kvm-x86-coco-7.4' of https://github.com/kvm-x86/linux into HEAD
KVM guest_memfd and CoCo changes for 7.4 - Take the invalidate lock when unbinding a dying file to fix a race with kvm_gmem_invalidate_end() that results in KVM leaving the MMU invalidation count elevated. Not taking the lock in the first place was pure misguided paranoid. - When binding a memslot, purge the bindings if storing the new memslot fails, as Xarray's "store range" API can subtly fail partway through. - Establish memslot<=>guest_memfd bindings *after* the memslot is ready, as the memslot becomes reachable the instant the binding is installed. - Don't grab a page refcount when reloading an SNP VMSA and stop returning a "struct page" from guest_memfd's PFN lookup altogether, to ensure that KVM doesn't hold any unnecessary refcounts (the lifetime of the page is ensured by MMU notifiers and similar hooks). Preventing unnecessary refcounts greatly simplifies supporting in-place SHARED=>PRIVATE conversions. - Add support for in-place PRIVATE<=>SHARED conversion, i.e. allow a single guest_memfd instance to be used for both PRIVATE and SHARED variations of CoCo guests' memory (though not at the same time). This eliminates the need to double allocate guest memory for TDX and SNP, and is basically a hard requirement for pKVM, which relies on being able to preserve memory contents on conversions. - Hack-a-fix a deadlock bug in ->error_remove_folio() by returning -EBUSY if the filemap invalidate lock is contended. Punt on a proper fix for the moment, as a proper fix likely involves reworking truncate_error_folio(). - Add a pile of selftests for in-place conversions. - Misc guest_memfd cleanups Conflicts in arch/arm64/kvm/mmu.c: convert to use s2vi. |
||
|
Paolo Bonzini
|
8edffbc1f6 |
Merge tag 'loongarch-kvm-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson into HEAD
LoongArch KVM changes for v7.4 1. Add separate vmid support. 2. Simplify MSI interrupt injection. 3. Add crash hypercall support. 4. Allow to set pv_feature until vCPU run. 5. Enable HW page table walk automatically in selftests. |
||
|
Paolo Bonzini
|
439d8f9cb0 |
Merge tag 'kvmarm-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 updates for 7.4
* New features
- pKVM state isolation: years after pKVM started being enabled
upstream, we finally get the basics in the form of state
isolation. The vcpu state is now off-limit for the host for
protected VMs.
- pKVM also grows a hypervisor-private memory allocator, allowing it
to deal with sub-page allocation, much like everywhere else.
- pKVM also lands a trace_printk() equivalent, which hopefully will
make debugging a bit easier.
- First shot at annotating portions of the KVM/arm64 code to be
extracted and built on s390. The real fun is yet to come.
- Allow pre-faulting of stage-2 canonical mappings from userspace,
joining x86 and s390 in implementing this feature.
- Add a reverse mapping infrastructure for shadow stage-2,
significantly improving performance when MMU notifiers kick in,
and avoiding full unmap of all shadow tables for this VM.
- Next step of GICv5 enablement, adding support for the IRS block,
and providing both SPI and LPI support. This is the first point
where KVM/arm64 can support an SMP GICv5 guest.
* Improvements
- As a preliminary to eventually enabling CCA, add a bunch of
infrastructure to KVM/arm64, splitting some of the basic
operations into individual backends (normal VM, normal pKVM VM,
protected pKVM VM). This allows CCA to later slot in without
polluting the rest of KVM with its firmware-driven nonsense.
- Make ID registers immutable from the point where a vcpu is able
to run, even if it fails to run. This prevents KVM from sampling
untrusted values.
- Restore some type checking to the EL2 object. KVM/arm64 originally
had some strong type checking between the kernel and the EL2
object, but lost it along the way when the private hypercall
interface was (somewhat pointlessly) converted to SMCCC. Type
checking is back, syscall style, and with added sparse goodies.
- Expand the FFA support to FFA_NOTIFICATION_*, and add some
sanitisation of the protocol that pKVM relays to the "trusted"
runtime.
* Fixes
- Make GICv3 RD assignment more robust to userspace nonsense
- Convert the remaining MMU fault handler backends to
kvm_s2_fault_vma_info and co, fixing a number of bugs in the
process.
- Fix the SError/SEA selftests to be both more robust and less
braindead.
- Fix a smattering of documentation bugs, some of them as a result
of our AI friends spitting out credible assertions based on
non-existing symbols.
- The usual collection of low impact bug fixes (SMCCC, timers, PMU).
|
||
|
Steffen Eiden
|
b62a0bf958 |
KVM: Replace CONFIG_KVM_MMIO with __KVM_NO_MMIO
Defining KVM_MMIO is not flexible enough for multi-KVM systems where one KVM implements MMIO but others do not. Therefore, remove KVM_MMIO from the config space and use the macro __KVM_NO_MMIO instead. Define __KVM_NO_MMIO for KVM/s390 and for HV-only 64-bit Book3s (aka pSeries/sPAPR) configurations. In particular, for PowerPC configurations: * KVM_BOOK3S_32 got KVM_MMIO via KVM_BOOK3S_32_HANDLER and KVM_BOOK3S_PR_POSSIBLE * KVM_BOOK3S_64 got KVM_MMIO via KVM_BOOK3S_PR_POSSIBLE if !KVM_BOOK3S_HV_POSSIBLE * KVM_BOOK3S_64 + KVM_BOOK3S_64_PR got KVM_MMIO via KVM_BOOK3S_PR_POSSIBLE * likewise for KVM_BOOK3S_64 + KVM_BOOK3S_64_PR + KVM_BOOK3S_64_HV * KVM_E500V2 and KVM_E500MC directly selected KVM_MMIO but KVM_BOOK3S_64 + KVM_BOOK3S_64_HV didn't and therefore needs __KVM_NO_MMIO. coalesced_mmio.c is now always compiled and linked. However, for architectures/configurations defining __KVM_NO_MMIO this effectively collapses to an empty object with no mmio code. No functional changes. Suggested-by: Marc Zyngier <maz@kernel.org> Signed-off-by: Steffen Eiden <seiden@linux.ibm.com> Link: https://patch.msgid.link/20260930173441.66762-4-seiden@linux.ibm.com [Simplify powerpc condition, add underscores. - Paolo] Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> |
||
|
Steffen Eiden
|
1499fd7cd2 |
KVM: Move architecture capability Kconfigs to header defines
With s390 hosting multiple independent KVM implementations within the same kernel build (native s390 KVM and arm64 on s390), global Kconfig selections force both implementations to inherit options that only apply to one of them. Replace global Kconfig options which only apply to one KVM implementation with architecture-specific opt-in defines in the respective <asm/kvm_host.h>. Only remove Kconfig symbols that would be different on both s390 KVM implementations - __KVM_HAVE_INVALID_WAKEUPS (formerly HAVE_KVM_INVALID_WAKEUPS) - __KVM_HAVE_MSI (formerly HAVE_KVM_MSI) - __KVM_HAVE_NO_POLL (formerly HAVE_KVM_NO_POLL) - __KVM_HAVE_READONLY_MEM (formerly HAVE_KVM_READONLY_MEM) - __KVM_HAVE_VCPU_RUN_PID_CHANGE (formerly HAVE_KVM_VCPU_RUN_PID_CHANGE) This allows each architecture and guest implementation to configure its supported capabilities independently in its kvm_host header and eliminates adding unnecessary fallback stubs across different KVM targets. Acked-by: Hendrik Brueckner <brueckner@linux.ibm.com> Signed-off-by: Steffen Eiden <seiden@linux.ibm.com> Link: https://patch.msgid.link/20261013121343.1997112-1-seiden@linux.ibm.com Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> |
||
|
Steffen Eiden
|
0d22ab9505 |
KVM: Make device name configurable
Allow KVM implementations to choose alternative device names. This is especially useful for architectures providing multiple KVM implementations simultaneously. Architectures providing multiple KVM implementations must compile the KVM common code once per implementation and mangle symbols. Suggested-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Steffen Eiden <seiden@linux.ibm.com> Link: https://patch.msgid.link/20260930173441.66762-2-seiden@linux.ibm.com Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> |
||
|
Marc Zyngier
|
62449bbaef |
Merge branch kvm-arm64/pre-faulting into kvmarm-master/next
* kvm-arm64/pre-faulting: : \ : Enable stage-2 pre-faulting in the canonical IPA space, implementing the : KVM_PRE_FAULT_MEMORY API. Patches courtesy of Lorenzo Stoakes, and based : on an initial work by Jack Thomson. : / KVM: selftests: Add nested pre-fault test for arm64 KVM: selftests: Add option for different backing in pre-fault tests KVM: selftests: Enable pre_fault_memory_test for arm64 Documentation: KVM: document arm64 KVM_PRE_FAULT_MEMORY KVM: arm64: Implement KVM_PRE_FAULT_MEMORY KVM: arm64: Pass walk flags to kvm_pgtable_get_leaf() KVM: arm64: Propagate EHWPOISON in kvm_s2_fault_pin_pfn() KVM: arm64: Size the stage-2 memcache from the fault MMU KVM: arm64: Propagate and use kvm_s2_fault_result on S2 fault KVM: arm64: Propagate and use mmu in s2fd when handling guest aborts KVM: arm64: Propagate and use esr in s2fd when handling guest aborts KVM: arm64: Use ESR helpers in guest abort handling arm64: Add ESR fault helpers KVM: Allow architectures to disallow pre-fault Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Paolo Bonzini
|
4e0ab192f2 |
Merge tag 'kvm-s390-next-7.4-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux into HEAD
KVM/vfio: Use file-based reference counting for KVM This series switches the KVM-VFIO interface and external consumers over to standard file-based reference counting, eliminating all external KVM symbol exports. Currently, VFIO integrates with KVM by looking up kvm_get_kvm_safe() and kvm_put_kvm() dynamically using symbol_get(), manually tracking module reference counts and storing a put_kvm function pointer in struct vfio_device. In the ARM64-on-s390 architecture, a second concurrent KVM module (kvm-arm64) is introduced alongside native KVM to host hardware-accelerated ARM64 guests. Having exported global symbols (like kvm_get_kvm/kvm_put_kvm) creates symbol conflicts and prevents clean coexistence of two KVM modules. Additionally, the file based counting simplifies the code and reuses the existing fs refcounting. Instead of passing raw KVM pointers and managing module symbols manually, the interface now passes the underlying VM file descriptor throughout VFIO and associated architecture subsystems. To safely extract the KVM instance from a file, an architecture-namespaced helper mechanism is introduced that verifies the file belongs to the expected KVM implementation before accessing its internal state. A back-pointer from the KVM instance to its associated file is maintained across its lifecycle so subsystems can safely acquire file references on demand. Finally, with VFIO, architecture page tracking, and device hooks converted to use file references, the remaining KVM reference- counting exports are restricted strictly to internal KVM modules. |
||
|
Fuad Tabba
|
8233000034 |
KVM: Fix comments that refer to the non-existent VCPU_RUN ioctl
Neither the VCPU_RUN in kvm_main.c nor the VCPU RUN in kvm_host.h names an ioctl that exists. The one that enters the guest is KVM_RUN, so refer to it by that name. Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev> Link: https://patch.msgid.link/20260915084441.3846947-2-fuad.tabba@linux.dev Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Steffen Eiden
|
c22295d0d7 |
KVM: Remove unused file_is_kvm
Remove the now unused file_is_kvm function. Signed-off-by: Steffen Eiden <seiden@linux.ibm.com> Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com> Acked-by: Janosch Frank <frankja@linux.ibm.com> Acked-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com> |
||
|
Steffen Eiden
|
00aa2c8a02 |
KVM: Restrict kvm_get_kvm/kvm_put_kvm export to internal KVM modules
Switch kvm_get_kvm, kvm_get_kvm_safe, and kvm_put_kvm from EXPORT_SYMBOL_GPL to EXPORT_SYMBOL_FOR_KVM_INTERNAL. These symbols are now only used within KVM itself. No external module needs them now that VFIO uses file-based reference counting. Remove the corresponding exemptions from the x86 KVM Makefile exports check. Suggested-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Steffen Eiden <seiden@linux.ibm.com> Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com> Acked-by: Janosch Frank <frankja@linux.ibm.com> Acked-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com> |
||
|
Steffen Eiden
|
5c3ea6d3e8 |
KVM/vfio: Use file-based reference counting for KVM
Replace manual module reference counting with file-based reference counting for KVM integration. Previously, VFIO used symbol_get() to obtain function pointers for kvm_get_kvm_safe() and kvm_put_kvm(), then manually tracked module references through these symbols. This approach required storing the put_kvm function pointer in each device and carefully managing symbol references. Pass struct file pointers instead of struct kvm pointers throughout the VFIO-KVM interface, leveraging the kernel's existing file reference counting via get_file()/get_file_active() and fput(). Convert the x86 page-track API and update s390 vfio to use file_to_kvm_<arch>(). This simplifies the code. Suggested-by: Jason Gunthorpe <jgg@nvidia.com> Co-developed-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Steffen Eiden <seiden@linux.ibm.com> Acked-by: Alex Williamson <alex@shazbot.org> Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com> Acked-by: Janosch Frank <frankja@linux.ibm.com> Acked-by: Sean Christopherson <seanjc@google.com> [borntraeger@linux.ibm.com remove _safe from vfio_device_get_kvm_safe] Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com> |
||
|
Steffen Eiden
|
8b5cb9769c |
KVM: Add file back-pointer to struct kvm
Add kvm->file as a no-reference back-pointer to the VM file in struct kvm. The pointer is set at VM creation time and cleared under WRITE_ONCE() in kvm_vm_release() before the file is freed. Callers storing the file must take their own reference. Co-developed-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Steffen Eiden <seiden@linux.ibm.com> Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com> Acked-by: Janosch Frank <frankja@linux.ibm.com> Acked-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com> |
||
|
Steffen Eiden
|
deaf98ff52 |
KVM: Introduce file_to_kvm_<arch>() infrastructure
Add a macro mechanism that generates an opt-in and arch-namespaced file_to_kvm_<arch>() to convert a file reference to a kvm object if the file handle represents a kvm handle. Activate for x86 and for s390 (native KVM only). Suggested-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Steffen Eiden <seiden@linux.ibm.com> Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com> Acked-by: Janosch Frank <frankja@linux.ibm.com> Acked-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com> |
||
|
Hao Zhang
|
ad06cf5338 |
KVM: guest_memfd: Avoid deadlock in error_remove_folio()
memory_failure() invokes ->error_remove_folio() while holding the
poisoned folio lock. KVM's guest_memfd callback then takes
mapping->invalidate_lock for read before invalidating guest mappings.
This can deadlock with paths that hold mapping->invalidate_lock for write
and then try to lock the same folio, e.g. truncation from MADV_REMOVE or
FALLOC_FL_PUNCH_HOLE. Because rwsems are fair, the problem is not limited
to guest_memfd punch-hole: a pending writer can also block future readers,
so ->error_remove_folio() must not block on the invalidate lock while
holding the poisoned folio lock.
Use filemap_invalidate_trylock_shared() in kvm_gmem_error_folio(). If the
invalidate lock is contended, fail the recovery instead of potentially
deadlocking the memory-failure path.
Fixes:
|
||
|
Sean Christopherson
|
41e005e2c5 |
KVM: WARN if vCPU creation is in-progress when locking all vCPUs
Now that KVM holds kvm->lock for the entirety of vCPU creation, from when created_vcpus is incremented until the new vCPU is fully onlined, WARN if the impossible happens and KVM somehow sees a discrepancy between the number of vCPUs "created" and "onlined". Signed-off-by: Sean Christopherson <seanjc@google.com> Tested-by: Jean-Christophe Guillain <jean-christophe@guillain.net> Tested-by: Naveen N Rao (AMD) <naveen@kernel.org> Message-ID: <20260921174445.911676-8-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> |
||
|
Sean Christopherson
|
700f8d2443 |
Revert "KVM: Check for duplicate vcpu_id as early as possible"
Now that KVM uses kvm_get_vcpu_by_id() to check for an existing vCPU ID before doing any meaningful work, which was made possible by holding kvm->lock for the entirety of vCPU creation, revert the now-redundant "early" vCPU ID tracking. The claims about the impact of kvm->vcpu_ids on the memory footprint were a wee bit wrong: the worst case scenario isn't 256 bytes per VM, it's 256 "unsigned longs" per VM, i.e. 2048 bytes per VM. Increasing the size of "struct kvm" by 2048 nearly doubled the total size on many architectures, and tripped x86's KVM_SANITY_CHECK_VM_STRUCT_SIZE, which was added to detect this *exact* scenario, where a single change significantly increased the size of "struct kvm". I.e. attempting to build KVM with CONFIG_DEBUG_KERNEL=n fails on x86 (the build failures got missed because all build bots apparently test only CONFIG_DEBUG_KERNEL=y kernels, and maintainers' test flows were similarly lacking). This reverts commit |
||
|
Sean Christopherson
|
c5866d42f7 |
KVM: Move check for existing vCPU ID to the top of vCPU creation
Now that kvm->lock is held for the entirety of vCPU creation, check for a
conflicting vCPU ID at the begnning of vCPU creation, before the arch
precreate() hook is invoked. This will allow reverting commit
|
||
|
Sean Christopherson
|
6418b715d5 |
KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock
When creating a vCPU, don't drop kvm->lock to when doing the bulk of actual
vCPU creation, as allowing multiple vCPUs to be created in parallel adds
significant complexity in KVM (as evidenced by the many related bugs), and
all known VMMs fully serialize vCPU creation. Remove all manually locking
of kvm->lock from kvm_arch_vcpu_{post,}create() for obvious reasons.
For many years, "everyone" has assumed that dropping kvm->lock was done for
performance reasons optimization, e.g. to allow userspace to create all
vCPUs concurrently for latency purposes. But as above, no known VMM does
that. Looking at the history of this code, before commit
|
||
|
Sean Christopherson
|
8264c0d73f |
KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress
Reject locking of all vCPUs if vCPU creation is in-progress, i.e. if the number of "created" vCPUs doesn't match the number of "onlined" vCPUs. It's simply not possible to guarantee that KVM has truly locked all vCPUs if one or more vCPUs are actively being created. Holding kvm->lock does prevent in-flight vCPUs from being fully onlined, but it's infeasible for common KVM to know whether or not that provides sufficient protection. In practice, this is likely a minor bug fix for the ARM and RISC-V usage of kvm_trylock_all_vcpus(), and a glorified nop for everything else. E.g. ARM's kvm_timer_vcpu_init() can race kvm_vm_ioctl_set_counter_offset() with respect to observing KVM_ARCH_FLAG_VM_COUNTER_OFFSET. Opportunistically drop x86's existing manual checks on vCPU creation being in-progress as all of x86's checks immediately precede or follow locking of all vCPUs. Leave arm64 and RISC-V alone for the moment, as their checks aren't as obviously redundant/equivalent. Signed-off-by: Sean Christopherson <seanjc@google.com> Tested-by: Jean-Christophe Guillain <jean-christophe@guillain.net> Tested-by: Naveen N Rao (AMD) <naveen@kernel.org> Message-ID: <20260921174445.911676-2-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> |
||
|
Lorenzo Stoakes (ARM)
|
caa63620f9 |
KVM: Allow architectures to disallow pre-fault
All existing architectures which implement KVM pre-fault (x86, s390) have mechanisms for disallowing pre-faulting. Currently these are open coded as part of kvm_arch_vcpu_pre_fault_memory(). Formalise this by moving them into a new kvm_arch_pre_fault_allowed() hook, which every architecture selecting CONFIG_KVM_GENERIC_PRE_FAULT_MEMORY must implement, returning an error code if the operation is disallowed or 0 otherwise. The hook is called early in the generic code, allowing architectures to disallow the operation prior to vCPU load. This is important, as kvm_vcpu_pre_fault_memory() is the only place where generic code can call vcpu_load() on a vCPU that has not yet been initialised. This lays the foundation for a future change which implements pre-faulting for arm64 which needs to disallow the mechanism for uninitialised vCPUs. Suggested-by: Oliver Upton <oupton@kernel.org> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Acked-by: Sean Christopherson <seanjc@google.com> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> Tested-by: Fuad Tabba <fuad.tabba@linux.dev> Link: https://patch.msgid.link/20260923-kvm-arm-prefault-v4-1-d4b0b4dfa8c3@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Christian Brauner
|
e2f0d6d456 |
fs: port ->setattr() to pass const mnt_idmap
Convert to const struct mnt_idmap. A mount's idmapping is immutable. The only thing that is allowed to be modified afterwards is the reference count and that is hidden behind mnt_idmap_get() and mnt_idmap_put(). Everything else only ever reads from the idmapping. This is the same model that struct cred uses and the idmapping is also rather sensitive. So make the idmap argument const wherever we can. The conversion is done from the bottom up so callers can continue to pass a non-const pointer to a const parameter until the conversion is finished. No functional changes. Link: https://patch.msgid.link/20260901-work-idmap-const-v1-24-54ccd48e100b@kernel.org Acked-by: Paul Moore <paul@paul-moore.com> Reviewed-by: Jan Kara <jack@suse.cz> Reviewed-by: Seth Forshee <sforshee@kernel.org> Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org> |
||
|
Zeng Chi
|
119e9db233 |
KVM: Don't pre-reserve xarray entries when storing empty/NULL attributes
Skip the xarray reservation loop when clearing all memory attributes, as storing NULL only erases the entry and never needs to allocate, so no reservation (and no cleanup of a failed one) is required in that case. Suggested-by: Sean Christopherson <seanjc@google.com> Cc: David Ballesteros <davimaba.v@proton.me> Signed-off-by: Zeng Chi <zengchi@kylinos.cn> Link: https://patch.msgid.link/20260921102442.1232375-1-zeng_chi911@163.com [sean: split to separate patch] Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
David Ballesteros
|
382e5d514b |
KVM: Ensure memory attributes xarray nodes are accounted to the caller's memcg
Explicitly instantiate the memory attributes xarray with XA_FLAGS_ACCOUNT
to ensure that all allocations are accounted to the memcg. Frustratingly,
memory allocations done in the "fastpath" do not honor the passed in gfp,
even for an explicit xa_reserve(). Only the rare, slow path __xas_nomem()
honors the original gfp. E.g.
xa_reserve(..., GFP_KERNEL_ACCOUNT)
|
-> ...
|
-> __xa_cmpxchg_raw()
|
-> xas_store() <== does not take @gfp
|
-> xas_create()
|
-> xas_alloc()
The bug was confirmed by observing that a process in a cgroup limited to
256 MiB grew radix_tree_node slab by ~512 MiB while its memory.current
stayed near 0.
Fixes:
|
||
|
Zeng Chi
|
277d3623d9 |
KVM: Don't treat reserved xarray entries as having memory attributes
kvm_vm_set_mem_attributes() reserves an xarray entry for every gfn in
the range before storing the new attributes, so that the store loop
can't fail partway through. If one of the reservations fails, e.g. with
-ENOMEM, the entries that were already reserved are left in the array.
That is harmless as far as xa_reserve() is concerned, as the reserved
entries read back as NULL via xa_load(), but it confuses the "does this
range have no attributes at all" check:
if (!attrs)
return !xas_find(&xas, end - 1);
A reserved entry is XA_ZERO_ENTRY, not NULL, and xas_find() returns it
as present. So a leftover reservation makes KVM report that a fully
shared range has attributes even though kvm_get_memory_attributes()
returns none for every gfn in the range. On x86, the next time
mixed-attribute tracking is recomputed for the range (memslot creation,
or a later attribute change that straddles the 2MiB page),
hugepage_has_attrs() treats a fully shared 2MiB range as mixed and
refuses to map it with a hugepage, until userspace happens to set
attributes on the range again.
Drop the shortcut and handle the !attrs case in the per-index loop,
using xas_next_entry() to find the next non-NULL entry. xas_next_entry()
is essentially an optimized xas_find(), so the effective change is that
the !attrs lookup now goes through xas_retry() like the attrs != 0 case,
i.e. reserved entries are skipped and retry entries restart the walk.
Don't check the index when no entry is found, as the xarray leaves the
xas index in a bogus state in that case; no entry simply means the rest
of the range has no attributes.
KVM never stores a non-NULL entry with a value of zero (clearing stores
NULL), but such an entry would be returned by xas_next_entry() and trip
the index check, so WARN if one is ever seen.
Fixes:
|
||
|
Sean Christopherson
|
537ec2b15f |
KVM: Let userspace disable per-VM mem attributes, enable per-gmem attributes
Allow the user to disable KVM_VM_MEMORY_ATTRIBUTES even when KVM supports PRIVATE and SHARED attributes, and expose gmem_in_place_conversion as a module parameter when per-VM attributes are supported. I.e. let userspace enable in-place PRIVATE<=>SHARED conversion of guest_memfd pages. Provide both a Kconfig option and a (conditional) module param so that deployments that use a custom kernel can fully disable per-VM tracking, while not forcing distros to ship two separate kernels in order to provide backwards compatibility for downstream users. Don't allow running VMs with mixed tracking for a given instance of KVM, i.e. disallow toggling the module param after KVM is loaded, as the extra complexity needed to handle per-VM behavior far outweighs any potential benefit. E.g. neither TDX nor SNP supports live migration, so in effect the requirement is that existing deployments that want to support both the old and the new models would need to tell their VMM which flavor of tracking to use. [ Xiaoyao: Define module_param only if CONFIG_KVM_VM_MEMORY_ATTRIBUTES is enabled ] Suggested-by: Xiaoyao Li <xiaoyao.li@intel.com> Co-developed-by: Ackerley Tng <ackerleytng@google.com> Signed-off-by: Ackerley Tng <ackerleytng@google.com> Tested-by: Shivank Garg <shivankg@amd.com> Reviewed-by: Fuad Tabba <tabba@google.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-22-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
31670c382e |
KVM: Move KVM_VM_MEMORY_ATTRIBUTES config definition to x86
Bury KVM_VM_MEMORY_ATTRIBUTES in x86 to discourage other architectures from adding support for per-VM memory attributes, because tracking private vs. shared memory on a per-VM basis is now deprecated in favor of tracking on a per-guest_memfd basis, and while RWX memory attributes are on the horizon, they too are expected to be x86-only. This will also allow modifying KVM_VM_MEMORY_ATTRIBUTES to be user-selectable (in x86) without creating weirdness in KVM's Kconfigs. Now that guest_memfd supports in-place conversions, it's entirely possible to run x86 CoCo VMs without support for KVM_VM_MEMORY_ATTRIBUTES. Leave the code itself in common KVM so that it's trivial to undo this change if new per-VM attributes do come along. Signed-off-by: Ackerley Tng <ackerleytng@google.com> Tested-by: Shivank Garg <shivankg@amd.com> Reviewed-by: Fuad Tabba <tabba@google.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-21-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Michael Roth
|
edcc3f4298 |
KVM: SEV: Make 'uaddr' parameter optional for KVM_SEV_SNP_LAUNCH_UPDATE
Make the source page for populating an SNP guest_memfd instance optional if in-place conversion/population is enabled. If KVM can convert the page in-place, then it's possible for guest memory to be initialized directly from userspace by mmap()'ing the guest_memfd and writing to it while the corresponding GPA ranges are in a 'shared' state, before converting them to the 'private' state expected by KVM_SEV_SNP_LAUNCH_UPDATE. Update the handling/documentation for KVM_SEV_SNP_LAUNCH_UPDATE to allow for 'uaddr' to be set to NULL when in-place conversion is enabled, which SNP_LAUNCH_UPDATE will then use to determine when it should/shouldn't copy in data from a separate memory location. Continue to enforce non-NULL when PRIVATE is tracked per-VM, not per-guest_memfd. [ Sean: Moved condition to snp_launch_update ] Signed-off-by: Michael Roth <michael.roth@amd.com> Signed-off-by: Ackerley Tng <ackerleytng@google.com> Tested-by: Shivank Garg <shivankg@amd.com> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-19-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Ackerley Tng
|
cb0544296a |
KVM: guest_memfd: Zero page while getting pfn
Move the folio initialization logic from kvm_gmem_get_pfn() into __kvm_gmem_get_pfn() to also zero pages if the page is to be used in kvm_gmem_populate(). With in-place conversion, the existing data in a guest_memfd page can be populated into guest memory through platform-specific ioctls. Without first zeroing the page obtained using __kvm_gmem_get_pfn(), it might contain uninitialized host memory, which would leak to the guest if the populate completes. guest_memfd pages are zeroed at most once in the page's entire lifetime with guest_memfd, and that is tracked using the uptodate flag. Zeroing the page in __kvm_gmem_get_pfn() is chosen over zeroing in kvm_gmem_get_folio() since other flows, such as a future write() syscall, can get a page, write to the page and then set page uptodate without zeroing. There may be some performance penalty due to redundant zeroing, but this would pale in comparison to the cost of actually assigning the page to the VM. This aligns with the concept of zeroing before first use - the other place where zeroing happens is in kvm_gmem_fault_user_mapping(). On populate failure, the page is not re-zeroed, since on SNP, if firmware rejects a CPUID page, the expected CPUID values provided by firmware are returned to userspace via page contents. More generally, page contents may be modified on populate failure. Don't mark the page uptodate again after populating, since the page would already be marked uptodate before the post_populate() call. Signed-off-by: Ackerley Tng <ackerleytng@google.com> Tested-by: Shivank Garg <shivankg@amd.com> Reviewed-by: Fuad Tabba <tabba@google.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-18-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Ackerley Tng
|
856852ef21 |
KVM: guest_memfd: Return early if range already has requested attributes
Provide a function to check that a range has given attributes. Optimize setting memory attributes by returning early if all pages in the requested range already have the requested attributes. Signed-off-by: Ackerley Tng <ackerleytng@google.com> Tested-by: Shivank Garg <shivankg@amd.com> Reviewed-by: Fuad Tabba <tabba@google.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-17-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Ackerley Tng
|
6fa1dab756 |
KVM: guest_memfd: Call arch make_shared callback for to-shared conversion
When doing in-place conversion from PRIVATE to SHARED, immediately inform arch code of the conversion for all allocated pages/folios, e.g. so that arch code can put hardware metadata tables in the correct state. Eagerly updating the table for to SHARED conversions avoids having to implement on-demand updates, e.g. when faulting in host userspace mappings. Skip the entire flow if the arch doesn't implement conversion callbacks, as getting folios from the filemap is noticeably expensive, especially when converting large chunks of memory. Deliberately don't eagerly update the metadata table on conversions from SHARED to PRIVATE, because assigning a page to a VM (versus "returning" it to the host) requires the exact GFN associated with the page, i.e would require walking the memslot bindings. And because KVM *must* do on-demand metadata updates when getting a PFN for KVM-internal usage, as that's the only time a relevant memslot binding is guaranteed to exist. Note! Inform arch code of the conversion within the protection of the invalidation sequence, to ensure that any existing mappings are dropped before hardware is updated, and to ensure that new mappings can't be established until after the conversion is complete. Signed-off-by: Ackerley Tng <ackerleytng@google.com> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-16-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Ackerley Tng
|
dcde2f853f |
KVM: guest_memfd: Ensure pages are not in use before conversion
When converting memory to private in guest_memfd, it is necessary to ensure that the pages are not currently being accessed by any other part of the kernel or userspace to avoid any current user writing to guest private memory. guest_memfd checks for any outstanding references to determine whether a page is still in use. The only expected references after unmapping the range requested for conversion are those that are held by guest_memfd itself. A folio will have outstanding references if it is present in a per-CPU lru_add fbatch. guest_memfd does not actually participate in LRU, but freshly-allocated folios are still added to the lru_add fbatch for batch LRU statistics processing. A folio may also have extra refcounts if it is on the mlock fbatch. These two known usages of the folio are handled by draining both the lru_add and mlock fbatches for the folio. After draining, if the refcount is still elevated, then there are truly outstanding references. If the page may be DMA-pinned, DMA is using it and hence there are outstanding references. Checking if the page may be DMA-pinned can have false positives, but that is only with a significant number of refcounts, at which point draining LRU is not going to move the needle - it can still be concluded that the folio has outstanding references. If the page is still mapped after guest_memfd tried to unmap it earlier in the conversion process, it also has outstanding references. Exit early to avoid unnecessary draining in these two cases. If an outstanding reference is detected, stop scanning, record the failing offset, and return immediately. Track the drain status to avoid repeatedly draining LRU caches across multiple folios while scanning the requested range. Update the kvm_memory_attributes2 structure to include an error_offset field. This allows KVM to report the exact offset where a conversion failed. If the safety check fails, return -EAGAIN and copy the error_offset back to userspace so that it can potentially retry the operation or handle the failure gracefully. Report error_offset if preallocating memory to track attributes fails with -ENOMEM as well, in which case the start offset of the range is returned. Update documentation to document the error_offset field and the possible -EAGAIN error. Suggested-by: David Hildenbrand <david@kernel.org> Co-developed-by: Vishal Annapurve <vannapurve@google.com> Signed-off-by: Vishal Annapurve <vannapurve@google.com> Signed-off-by: Ackerley Tng <ackerleytng@google.com> Tested-by: Shivank Garg <shivankg@amd.com> Reviewed-by: Fuad Tabba <tabba@google.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org> Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-15-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Ackerley Tng
|
799c9fadc6 |
KVM: guest_memfd: Add base support for KVM_SET_MEMORY_ATTRIBUTES2
Add a new ioctl (and matching struct), KVM_SET_MEMORY_ATTRIBUTES2, using the same base ioctl number (0xd2), but with R/W semantics for the kernel instead of just read semantics. "Officially" documenting that KVM writes to the payload will allow KVM to support partial/incremental conversions, instead of all-or-nothing updates (which requires complex unwinding), by recording the failing offset if an error occurs. Opportunistically add a new struct as well, even though KVM could squeeze the error offset into "struct kvm_memory_attributes", as there's no cost to doing so in practice. Pad the struct with a pile of extra space to try and avoid ending up with "struct kvm_memory_attributes3" in the future. Use the same layout for the fields common to version 1 of the struct, e.g. to ease upgrading userspace, and to provide flexibility if KVM ever adds support for KVM_SET_MEMORY_ATTRIBUTES2 at VM scope. Introduce KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES to advertise the availability of the KVM_SET_MEMORY_ATTRIBUTES2 ioctl. Update the KVM API documentation to define the new ioctl and its behavior, and add the necessary UAPI definitions and capability checks. The process of setting memory attributes has a clear point of no return because, for CoCo VMs, zapping stage 2 page tables is a destructive operation. Unlike regular VMs, where re-faulting pages into the stage 2 page tables merely incurs a performance penalty, CoCo guests must (re-):accept pages after every fault. To preserve CoCo security guarantees, guests will not accept pages they did not explicitly request faults for. Consequently, during memory conversions, any operation that could cause the process to abort must be completed before the stage 2 page tables are zapped. Zap only the ranges that are not already in the requested state to avoid inadvertently destroying (CoCo) data. ARM CCA guests will try to mark the entire DRAM as private at boot. If there are no shared pages at all, the to-private conversion can be skipped, but the existence of a single shared page would require the conversion process to proceed, and if it proceeds, zapping both shared and private pages would destroy data and break the guest. Suggested-by: Michael Roth <michael.roth@amd.com> Suggested-by: Suzuki K Poulose <suzuki.poulose@arm.com> Co-developed-by: Vishal Annapurve <vannapurve@google.com> Signed-off-by: Vishal Annapurve <vannapurve@google.com> Co-developed-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Ackerley Tng <ackerleytng@google.com> Tested-by: Shivank Garg <shivankg@amd.com> Reviewed-by: Fuad Tabba <tabba@google.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-14-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Ackerley Tng
|
2f7f4e1028 |
KVM: guest_memfd: Pass mapping type filter to invalidation helper
Accept the mapping type filter as a parameter in the invalidation start helper instead of querying it internally. This allows callers to specify which mappings (shared, private, or both) should be invalidated. In the next patch, the conversion process will use this new parameter to invalidate mappings only when they're different from the target state of the conversion, i.e. invalidate only shared mappings on a shared to private conversion and not both. No functional change intended. Signed-off-by: Ackerley Tng <ackerleytng@google.com> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-13-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Ackerley Tng
|
2e646bcfcf |
KVM: guest_memfd: Always fault from guest_memfd if in-place conversion is enabled
If a guest_memfd memslot is created but the guest_memfd does not have the GUEST_MEMFD_FLAG_MMAP set, KVM still fulfils guest faults by looking up the memslot's userspace_addr. Set KVM_MEMSLOT_GMEM_ONLY if in-place conversion is enabled so that the guest_memfd's memory will be used for both shared and private memory. With in-place conversion, guest_memfd will be the only backing memory for the memslot. No validation is performed to require userspace_addr to be a mapping from the associated guest_memfd because even after validation, userspace is free to remap something else at the provided userspace_addr. userspace_addr will still be used by functions like kvm_read_guest(), and if userspace_addr does not match up with the corresponding memory in the memslot's guest_memfd (whether userspace_addr points to the wrong offset or some non-guest_memfd memory, etc), that is a user error. Requiring both shared and private memory to come from the only associated guest_memfd simplifies invalidation in stage 2 page tables. On a PUNCH_HOLE operation on a guest_memfd, the invalidation is now guaranteed to be invalidating only memory mapped from the given guest_memfd. Suggested-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Ackerley Tng <ackerleytng@google.com> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-12-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
e088c7f86b |
KVM: guest_memfd: Invalidate both SHARED and PRIVATE mappings for in-place conversions
When removing one or more folios from a guest_memfd instance, invalidate both SHARED and PRIVATE mappings if in-place conversion is enabled, because stating the obvious, KVM needs to ensure that all mappings to the folio(s) are dropped. Opportunistically rename the helper to capture that it returns a filter for all gfns in anticipation of zapping only the previous mapping types on conversion. I.e. when doing in-place conversion to PRIVATE, only SHARED mappings need to be zapped (ignoring that KVM would ideally not invalidate ranges whose attributes aren't changing in the first place). Note, precisely zapping only the possible mapping types when in-place conversion is disabled is important for functional correctness, not just for performance. Specifically, if KVM zaps both when SHARED vs. PRIVATE is tracked per-VM, then a PUNCH_HOLE operation on a PRIVATE guest_memfd will incorrectly zap SHARED mappings that have nothing to do with that gmem instance (because they're mapped via a VMA, not a gmem fd). The incorrect over-zapping of SHARED memory that doesn't belong to the gmem fd requesting the zapping will be resolved in a later patch, where, if in-place conversion is enabled, KVM will use both shared and private memory from the guest_memfd. If both shared and private memory are from the guest_memfd, invalidation will only zap memory belonging to the given gmem instance. Signed-off-by: Ackerley Tng <ackerleytng@google.com> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-11-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
ee1168fb73 |
KVM: guest_memfd: Stub in ability to enable in-place shared<=>private conversion
Stub in global variable to enable in-place guest_memfd private<=>shared memory conversion, which will eventually be exposed to userspace via a module param, and wire up the __kvm_is_private_gfn() static call to the guest_memfd version when in-place conversion is enabled, i.e. when gmem is the sole authority on private vs. shared memory. Co-developed-by: Ackerley Tng <ackerleytng@google.com> Signed-off-by: Ackerley Tng <ackerleytng@google.com> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org> Cc: Fuad Tabba <tabba@google.com> Cc: Xiaoyao Li <xiaoyao.li@intel.com> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-9-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
8ab5719fbc |
KVM: Provide generic interface for checking memory private/shared status
Introduce a generic kvm_is_private_gfn() interface using a static call to determine if a GFN is private. This allows the implementation for checking a GFN's private/shared status to be set at runtime. In preparation for choosing implementations between a guest_memfd lookup and the existing VM attribute lookup, rename the existing VM-attribute-based check to kvm_vm_is_private_gfn() to emphasize that it looks up VM attributes. Signed-off-by: Ackerley Tng <ackerleytng@google.com> Tested-by: Shivank Garg <shivankg@amd.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Fuad Tabba <tabba@google.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-8-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Ackerley Tng
|
98fd6e30bd |
KVM: Rename kvm_mem_is_private() to kvm_is_private_gfn()
Rename kvm_mem_is_private() to kvm_is_private_gfn() to prepare for in-place conversion, where there will be two lookup functions, kvm_vm_is_private_gfn() and kvm_gmem_is_private_gfn(). This renaming allows consistent prefixing of "vm" vs "gmem" for kvm_*_is_private_gfn(), as opposed to kvm_gmem_mem_is_private(), which looks like a typo. Suggested-by: Sean Christopherson <seanjc@google.com> Signed-off-by: Ackerley Tng <ackerleytng@google.com> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-7-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
a84ef89cbf |
KVM: Rename memory attribute APIs to prepare for in-place gmem conversion
Rename memory attribute APIs to add a "vm_" in the name in anticipation of moving PRIVATE tracking into guest_memfd, to allow in-place conversion between SHARED and PRIVATE. At that point, there will effectively be two (potential) sources of memory attributes: the VM and guest_memfd. kvm_vm_set_mem_attributes() already has "vm" in the name to indicate that it is a VM ioctl. Rename it to kvm_set_vm_mem_attributes() to show that it is setting the VM's memory attributes. (Drop the VM-ioctl scoping since it is a helper local to the file.) Update the accompanying trace function to match. No functional change intended. Signed-off-by: Ackerley Tng <ackerleytng@google.com> Tested-by: Shivank Garg <shivankg@amd.com> Reviewed-by: Fuad Tabba <tabba@google.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-6-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
c2ae2bc1a3 |
KVM: Enumerate support for PRIVATE memory iff kvm_arch_has_private_mem is defined
Explicitly guard reporting support for KVM_MEMORY_ATTRIBUTE_PRIVATE based on kvm_arch_has_private_mem being #defined in anticipation of tracking PRIVATE vs. SHARED state per-guest_memfd, not per-VM (to allow in-place conversion). guest_memfd support for memory attributes is expected to be unconditional to avoid yet more macros (all architectures that support guest_memfd are expected to use per-gmem attributes at some point), at which point enumerating support KVM_MEMORY_ATTRIBUTE_PRIVATE based solely on memory attributes being supported by KVM at-large would result in a system-scope check (NULL @kvm) over-reporting support on arm64. Give architectures full control over overriding the default definition of kvm_arch_has_private_mem() by removing the coupling with CONFIG_KVM_VM_MEMORY_ATTRIBUTES. In a later patch, kvm_arch_has_private_mem() will be defined based on whether architectural features are compiled in, and made orthogonal to CONFIG_KVM_VM_MEMORY_ATTRIBUTES. Signed-off-by: Ackerley Tng <ackerleytng@google.com> Tested-by: Shivank Garg <shivankg@amd.com> Reviewed-by: Fuad Tabba <tabba@google.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-5-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
44e17d0a36 |
KVM: Rename KVM_GENERIC_MEMORY_ATTRIBUTES to KVM_VM_MEMORY_ATTRIBUTES
Rename the per-VM memory attributes Kconfig to make it explicitly about per-VM attributes in anticipation of adding memory attributes support to guest_memfd, at which point it will be possible (and desirable) to have memory attributes without the per-VM support, even in x86. No functional change intended. Signed-off-by: Ackerley Tng <ackerleytng@google.com> Tested-by: Shivank Garg <shivankg@amd.com> Reviewed-by: Fuad Tabba <tabba@google.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-4-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
1ae5be958f |
KVM: guest_memfd: Introduce per-gmem attributes, use to guard user mappings
Start plumbing in guest_memfd support for in-place private<=>shared conversions by tracking attributes via a maple tree. KVM currently tracks private vs. shared attributes on a per-VM basis, which made sense when a guest_memfd _only_ supported private memory, but tracking per-VM simply can't work for in-place conversions as the shared/private status of a given page needs to be per-gmem_inode, not per-VM. Use the filemap invalidation lock to protect the maple tree, as taking the lock for read when faulting in memory (for userspace or the guest) isn't expected to result in meaningful contention, and using a separate lock would add significant complexity (avoiding deadlock is quite difficult). In kvm_gmem_get_pfn(), drop the folio refcount before releasing filemap_invalidate_lock(). This ensures that a competing conversion request from userspace (to be added in a later patch), which also takes the filemap_invalidate_lock(), will never see an elevated refcount due to kvm_gmem_get_pfn(). Co-developed-by: Vishal Annapurve <vannapurve@google.com> Signed-off-by: Vishal Annapurve <vannapurve@google.com> Co-developed-by: Fuad Tabba <tabba@google.com> Signed-off-by: Fuad Tabba <tabba@google.com> Co-developed-by: Ackerley Tng <ackerleytng@google.com> Signed-off-by: Ackerley Tng <ackerleytng@google.com> Tested-by: Shivank Garg <shivankg@amd.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-3-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
fa7b8b1bb5 |
KVM: guest_memfd: Use kvm_mem_is_private() when populating guest_memfd memory
Use kvm_mem_is_private() when populating guest_memfd instead of using an open coded equivalent. In addition to simplifying the populate code *now*, this avoids the need to provide a range-based gmem lookup API in the future as well. No functional change intended. Suggested-by: Xiaoyao Li <xiaoyao.li@intel.com> Signed-off-by: Ackerley Tng <ackerleytng@google.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-2-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
54623a16bb |
KVM: guest_memfd: Optimize away conversion overheads via dead-code elimination
Add and use kvm_arch_has_gmem_convert() to guard guest_memfd's invocation of arch hooks related to converting memory between private and shared, as only one half of the x86 CoCo duo needs the runtime hooks (any pre-work is pure overhead for TDX). At this exact moment, the overhead is negligible, but that will change when in-place conversion comes along, at which point to-shared conversions will "need" to find all affected folios prior to calling into arch code. In quotes because very technically that work could be pushed to arch code, but that would bleed guest_memfd details into arch code and would be far worse than adding yet another kvm_arch_has... hook. Opportunistically provide the kvm_arch_gmem_make_private() declaration, and rely on dead-code elimination to eliminate the call to non-existent code when CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT=n. Reported-by: Binbin Wu <binbin.wu@linux.intel.com> Closes: https://lore.kernel.org/all/1ec08cd8-3072-4753-ad5e-cd34956647f8@linux.intel.com Suggested-by: Ackerley Tng <ackerleytng@google.com> Signed-off-by: Ackerley Tng <ackerleytng@google.com> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Tested-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://patch.msgid.link/20260910-gmem-inplace-conversion-v13-1-dd6fbf94f4e1@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |