Lines Matching defs:page
51 #include <asm/page.h>
101 * where the hardware walks 2 page tables:
542 * Flush TLB when accessed/dirty states are changed in the page tables,
543 * to guarantee consistency between TLB and page tables.
570 struct page *page;
589 * by a refcounted page, the refcount is elevated.
591 page = kvm_pfn_to_refcounted_page(pfn);
592 WARN_ON_ONCE(page && !page_count(page));
631 * Capture the dirty status of the page, so that it doesn't get
655 * Prevent page table teardown by making any free-er wait during
676 * OUTSIDE_GUEST_MODE and proceed to free the shadow page table.
745 * KVM is not shadowing any guest page tables, so the "guest access
749 * is shadowing a guest huge page with small pages, the guest access
751 * page.
767 "access mismatch under %s page %llx (expected %u, got %u)\n",
772 "gfn mismatch under %s page %llx (expected %llx, got %llx)\n",
786 * Return the pointer to the large page information for a given gfn,
787 * handling slots that are not large page aligned.
845 * If it's possible to replace the shadow page with an NX huge page,
846 * i.e. if the shadow page is the only thing currently preventing KVM
847 * from using a huge page, add the shadow page to the list of "to be
848 * zapped for NX recovery" pages. Note, the shadow page can already be
849 * on the list if KVM is reusing an existing shadow page, i.e. if KVM
850 * links a shadow page at multiple points.
1212 * spte write-protection is caused by protecting shadow page table.
1219 * shadow page.
1299 * Used when we do not need to care about huge page mappings.
1326 * protect the page if the D-bit isn't supported.
1364 * We need to care about huge page mappings: e.g. during dirty logging we may
1799 } page[KVM_PAGE_ARRAY_NR];
1810 if (pvec->page[i].sp == sp)
1813 pvec->page[pvec->nr].sp = sp;
1814 pvec->page[pvec->nr].idx = idx;
1918 * page using the current MMU context.
1934 * sync a shadow page for a different MMU context, e.g. if the role
1974 * bits are recognized, i.e. until the guest flushes or page faults on
2022 i < pvec.nr && ({ sp = pvec.page[i].sp; 1;}); \
2032 struct kvm_mmu_page *sp = pvec->page[n].sp;
2033 unsigned idx = pvec->page[n].idx;
2055 WARN_ON_ONCE(pvec->page[0].idx != INVALID_INDEX);
2057 sp = pvec->page[0].sp;
2141 * page may already exist and syncing it needs the vCPU pointer in
2142 * order to read guest page tables. Direct shadow pages are never
2164 * If the guest is creating an upper-level page, zap
2166 * the guest is using recursive page tables, in all
2168 * page and is installing a completely unrelated page.
2170 * upper-level page will be write-protected.
2187 * The page is good, but is stale. kvm_sync_page does
2189 * it doesn't write-protect the page or mark it synchronized!
2195 * If the sync fails, the page is zapped. If so, break
2223 /* Caches used when allocating a new shadow page. */
2317 * shadow each guest page table with multiple shadow page tables, which
2320 * Specifically, to shadow the guest's page directory (which covers a
2321 * 4GiB address space), KVM uses 4 PAE page directories, each mapping
2325 * To shadow the guest's page tables (which each map a 4MiB region), KVM
2326 * uses 2 PAE page tables, each mapping a 2MiB region. For these,
2331 * PDPTEs; those 4 PAE page directories are pre-allocated and their
2591 * the active page list. See list_del() in the "else" case of
2601 * Remove the active root from the active page list, the root
2647 * the page tables and see changes to vcpu->mode here. The barrier
2652 * guest mode and/or lockless shadow page table walks.
2677 * Don't zap active root pages, the page itself can't be freed
2719 * page is available, while the caller may end up allocating as many as
2724 * page fault paths.
2796 * were marked unsync (or if there is no shadow page), -EPERM if the SPTE must
2806 * Force write-protection if the page is being tracked. Note, the page
2814 * The page is not write-tracked, mark existing shadow pages unsync
2830 * TDP MMU page faults require an additional spinlock as they
2842 * may have since marked the page unsync. A false
2863 * before the page had been marked as unsync-ed, something like the
2871 * (GPTE being in the guest page table shadowed
2873 * This reads SPTE during the page table walk.
2881 * false and skips the page.
2925 * If we overwrite a PTE page pointer with a 2MB PMD, unlink
2975 struct page *pages[PTE_PREFETCH_NUM];
3072 * Note! The lookup can still race with modifications to host page tables, but
3092 * page fault steps have already verified the guest isn't writing a
3098 * Disable IRQs to prevent concurrent tear down of host page tables,
3099 * e.g. if the primary MMU promotes a P*D to a huge page and then frees
3100 * the original page table.
3106 * a huge page _during_ this walk. Re-reading the entry could send the
3108 * value) and then p*d_offset() walks into the target huge page instead
3109 * of the old page table (sees the new value).
3229 * We cannot overwrite existing page tables with an NX
3230 * large page, as the leaf could be executable.
3278 * caused mmio page fault and treat it as mmio access.
3302 * touching the shadow page tables as attempting to install an
3325 * reach the common page fault handler if the SPTE has an invalid MMIO
3335 * 1. The shadow page table entry is not present and A/D bits are
3339 * bits for L2 and employ access tracking, but the fast page fault
3341 * 2. The shadow page table entry is present, the access is a write,
3399 * Returns the last level spte pointer of the shadow page walk for the given
3470 * KVM only supports fixing page faults outside of MMU lock for
3495 * first page into the dirty-bitmap in
3499 * Instead, we let the slow page fault path create a
3513 * Currently, fast page fault only works for direct mapping
3514 * since the gfn is not stable for indirect shadow page. See
3809 * Do we shadow a long mode page table? If so we need to
3810 * write-protect the guests page table root.
3825 * We shadow a 32 bit page table. This may be a legacy 2-level
3826 * or a PAE 3-level page table. In either case we need to be aware that
3827 * the shadow page table may be a PAE or a long mode page table.
3860 * If shadowing 32-bit non-PAE page tables, each PAE page
3861 * directory maps one quarter of the guest's non-PAE page
3862 * directory. Othwerise each PAE page direct shadows one guest
3863 * PAE page directory so that quadrant should be 0.
3907 * of levels for the shadow page tables, e.g. all MMUs are 4-level or
3966 * The read barrier orders the CPU's read of SPTE.W during the page table
3970 * any guest page table changes are not guaranteed to be visible anyway
3973 * before any actual changes to the page tables are made. The comments
4060 * page tables, because cr2 is a nGPA while the cache stores GPAs.
4174 * If the page table is zapped by other cpus, let CPU fault again on
4190 * guest is writing the page which is write tracked which can
4191 * not be fixed by page fault handler.
4260 * Retry the page fault if the gfn hit a memslot that is being deleted
4276 * If the APIC access page exists but is disabled, go directly
4291 return RET_PF_CONTINUE; /* *pfn has correct page already */
4307 * get a page and a fatal signal, i.e. SIGKILL, is pending.
4337 * Returns true if the page fault is stale and needs to be retried, i.e. if the
4350 * Roots without an associated shadow page are considered invalid if
4632 * It's possible that the cached previous root page is obsolete because
4648 * the shadow page tables.
4653 * If this is a direct root page, it doesn't have a write flooding
4725 /* no rsvd bits for 2 level 4K page table entries */
4737 /* 36bits PSE 4MB page */
4740 /* 32 bits PSE 4MB page */
4751 rsvd_bits(13, 20); /* large page */
4776 rsvd_bits(13, 20); /* large page */
4813 /* large page */
4846 * the page table on host is the shadow page table for the page
4855 /* KVM doesn't use 2-level page tables for the shadow MMU. */
4894 * the direct page table on host, use as much mmu features as
4925 * is the shadow page table for intel nested guest.
4999 * - A user page is accessed
5020 * key violations are reported through a bit in the page fault error code.
5024 * CR0, EFER, CPL), and on other bits of the error code and the page tables.
5027 * page tables and the machine state:
5030 * - PK is always zero if U=0 in the page tables
5034 * code (minus the P bit) and the page table's U bit form an index into the
5067 * instruction fetch and is to a user page.
5379 * L2 page tables are never shadowed, so there is no need to sync
5386 * L1's nested page tables (e.g. EPT12). The nested translation
5388 * L2's page tables as the first level of translation and L1's
5389 * nested page tables as the second level of translation. Basically
5507 * have an associated shadow page, as it's impossible to determine if
5555 * Assume that the pte write on a page table of the same type
5575 * If we're seeing too many writes to a page, it may no longer be a page table,
5576 * or we may be forking, in which case it is better to unmap the page.
5582 * it can become unsync, then the guest page is not write-protected.
5593 * indicate a page is not used as a page table.
5658 * If we don't have indirect shadow pages, it means no page is
5738 * was due to a RO violation while translating the guest page.
5740 * paging in both guests. If true, we simply unprotect the page
5751 * optimistically try to just unprotect the page and let the processor
5752 * re-execute the instruction that caused the page fault. Do not allow
5757 * explicitly shadowing L1's page tables, i.e. unprotecting something
5891 * the kernel is not. But, KVM never creates a page size greater than
5968 struct page *page;
5994 page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_DMA32);
5995 if (!page)
5998 mmu->pae_root = page_address(page);
6002 * get the CPU to treat the PDPTEs as encrypted. Decrypt the page so
6059 * No obsolete valid page exists before a newly created page
6067 * pages. Skip the bogus page, otherwise we'll get stuck in an
6068 * infinite loop if the page gets put back on the list (again).
6094 * Kick all vCPUs (via remote TLB flush) before freeing the page tables
6095 * to ensure KVM is not in the middle of a lockless shadow page table
6140 * Notify all vcpus to reload its shadow page table and flush TLB.
6141 * Then all vcpus will switch to new shadow page table with the new
6145 * otherwise, vcpu would purge shadow page but miss tlb flush.
6300 * to split a single huge page. Calculating how many are actually needed
6353 * Note, huge page splitting always uses direct shadow pages, regardless
6354 * of whether the huge page itself is mapped by a direct or indirect
6355 * shadow page, since the huge page region itself is being directly
6389 * page is aliased by multiple sptes with the same access
6394 * However, if a given SPTE points to a lower level page table,
6395 * that lower level page table may only be partially populated.
6397 * huge page. Unmapping guest memory always requires a TLB flush
6582 * We cannot do huge page mapping for indirect shadow pages,
6584 * tdp; such shadow pages are synced with the page table in
6585 * the guest, and the guest page table is using 4K page size
7040 * NX huge page is expected to be relatively small compared to
7061 * is much cheaper than zapping the page.
7066 * it is becoming nonzero, the page will be zapped unnecessarily.