Grab vmobjlocks with RW_DUPOK in vm_obj_wire() to silence WITNESS

The drm subsystem implements graphics buffers as uvm objects backed by
anonymous memory, thus drm locks and aobj locks share the sam

Grab vmobjlocks with RW_DUPOK in vm_obj_wire() to silence WITNESS

The drm subsystem implements graphics buffers as uvm objects backed by
anonymous memory, thus drm locks and aobj locks share the same "vmobjlock"
type.

uvm_obj_wire() is only called from sys/dev/pci/drm/, so instead of changing
drm's lock init/alloc routines to mark allow duplicate locks in general,
enter uvm's vmobjlock with RW_DUPOK in this function to allow duplicate
lock types per thread in this specific call path alone.

Fixes the following WITNESS report when booting/starting X (as seen already
in other unrelated bugs@ reports):

wsdisplay0: screen 1-5 added (std, vt100 emulation)
witness: acquiring duplicate lock of same type: "&uobj->vmobjlock"
1st uobjlk
2nd uobjlk
Starting stack trace...
witness_checkorder(fffffd83b625f9b0,9,0) at witness_checkorder+0x8ac
rw_enter(fffffd83b625f9a0,1) at rw_enter+0x68
uvm_obj_wire(fffffd843c39e948,0,40000,ffff800033b70428) at uvm_obj_wire+0x46
shmem_get_pages(ffff800008008500) at shmem_get_pages+0xb8
__i915_gem_object_get_pages(ffff800008008500) at __i915_gem_object_get_pages+0x6d
i915_gem_fault(ffff800008008500,ffff800033b707c0,10009b000,a43d6b1c000,ffff800033b70740,1,35ba896911df1241,ffff8000000aa078,ffff8000000aa178) at i915_gem_fault+0x203
drm_fault(ffff800033b707c0,a43d6b1c000,ffff800033b70740,1,0,0,7eca45006f70ee0,ffff800033b707c0) at drm_fault+0x156
uvm_fault(fffffd843a7cf480,a43d6b1c000,0,2) at uvm_fault+0x179
upageflttrap(ffff800033b70920,a43d6b1c000) at upageflttrap+0x62
usertrap(ffff800033b70920) at usertrap+0x129
recall_trap() at recall_trap+0x8
end of kernel
end trace frame: 0x7f7ffffdc7c0, count: 246
End of stack trace.

Input kettenis
OK mpi

show more ...

Call uvm_pglistfree(9) instead of uvm_pmr_freepageq().

There is no functionnal change as the former is just a wrapper around the
latter. However upper layer of UVM do not need to mess with the inte

Call uvm_pglistfree(9) instead of uvm_pmr_freepageq().

There is no functionnal change as the former is just a wrapper around the
latter. However upper layer of UVM do not need to mess with the internals
of the page allocator.

This will also help when a page cache will be introduced to reduce contention
on the global mutex serializing acess to pmemrange's data.

ok kettenis@, kn@, tb@

show more ...

Use a per-UVM object lock to serialize the lower part of the fault handler.

Like the per-amap lock the `vmobjlock' is principally used to serialized
access to objects in the fault handler to allow f

Use a per-UVM object lock to serialize the lower part of the fault handler.

Like the per-amap lock the `vmobjlock' is principally used to serialized
access to objects in the fault handler to allow faults occurring on
different CPUs and different objects to be processed in parallel.

The fault handler now acquires the `vmobjlock' of a given UVM object as
soon as it finds one. For now a write-lock is always acquired even if
some operations could use a read-lock.

Every pager, corresponding to a different kind of UVM object, now expect
the UVM object to be locked and some operations, like *_get() return it
unlocked. This is enforced by assertions checking for rw_write_held().

The KERNEL_LOCK() is now pushed to the VFS boundary in the vnode pager.

To ensure the correct amap or object lock is held when modifying a page
many uvm_page* operations are now asserting for the "owner" lock.
However, fields of the "struct vm_page" are still being protected by the
global `pageqlock'. To prevent lock ordering issues with the new
`vmobjlock' and to reduce differences with NetBSD this lock is now taken
and released for each page instead of around the whole loop.

This commit does not remove the KERNEL_LOCK/UNLOCK() dance. Unlocking
will follow if there is no fallout.

Ported from NetBSD, tested by many, thanks!

ok kettenis@, kn@

show more ...

Sprinkle uvm_obj_destroy() over UVM object recycling code.

For now, only assert that the tree of pages is empty in uvm_obj_destroy().
This will soon be used to free the per-UVM object lock.

While h

Sprinkle uvm_obj_destroy() over UVM object recycling code.

For now, only assert that the tree of pages is empty in uvm_obj_destroy().
This will soon be used to free the per-UVM object lock.

While here call uvm_obj_init() when new vnodes are allocated instead of
in uvn_attach(). Because vnodes and there associated UVM object are
currently never freed, it isn't easy to know where/when to garbage
collect the associated lock. So simply check that the reference of a
given object is 0 when uvn_attach().

Tested by many as part of a bigger diff.

ok kettenis@

show more ...

Introduce a dummy uvm_obj_destroy() interface. This function will be
used in the near future (by mpi@) to improve the locking for uvm objects.
Introducing this function now will me allow me to call

Introduce a dummy uvm_obj_destroy() interface. This function will be
used in the near future (by mpi@) to improve the locking for uvm objects.
Introducing this function now will me allow me to call it in the
appropriate place in the drm code.

ok mpi@, jsg@

show more ...

Introduce dummy pagers for 'special' subsystems using UVM objects.

Some pmaps (x86, hppa) and the buffer cache rely on UVM objects to allocate
and manipulate pages. These objects should not be mani

Introduce dummy pagers for 'special' subsystems using UVM objects.

Some pmaps (x86, hppa) and the buffer cache rely on UVM objects to allocate
and manipulate pages. These objects should not be manipulated by uvm_fault()
and do not currently require the same locking enforcement.

Use the dummy pagers to explicitly document which UVM functions are meant to
manipulate UVM objects (uobj) that do not need the upcoming `vmobjlock' and
instead still rely on the KERNEL_LOCK().

Tested by many as part of a larger diff.

ok kettenis@, beck@

show more ...

Change the prefix of UVM object functions to match NetBSD's.

For example uvm_objinit() becomes uvm_obj_init(). Reduce differences
between the trees and help porting new functions needed for UVM obj

Change the prefix of UVM object functions to match NetBSD's.

For example uvm_objinit() becomes uvm_obj_init(). Reduce differences
between the trees and help porting new functions needed for UVM object
locking.

No functionnal change.

show more ...

Grab the `pageqlock' before calling uvm_pageclean() as intended.

Document which global data structures require this lock and add some
asserts where the lock should be held.

Some code paths are stil

Grab the `pageqlock' before calling uvm_pageclean() as intended.

Document which global data structures require this lock and add some
asserts where the lock should be held.

Some code paths are still incorrect and should be revisited.

ok jmatthew@

show more ...

Constify and use C99 initializer for "struct uvm_pagerops".

While here put some KERNEL_ASSERT_LOCKED() in the functions called from
the page fault handler. The removal of locking of `uobj' will nee

Constify and use C99 initializer for "struct uvm_pagerops".

While here put some KERNEL_ASSERT_LOCKED() in the functions called from
the page fault handler. The removal of locking of `uobj' will need to be
revisited and these are good indicator that something is missing and that
many comments are lying.

ok kettenis

show more ...

Include <sys/systm.h> directly instead of relying on uvm_map.h to pull it.

Add uvm_objfree function to free all pages in a uvm_obj in one go.
Use this in the buffer cache to free all the pages from a buffer,
resulting in a considerable speedup when throwing away pages from

Add uvm_objfree function to free all pages in a uvm_obj in one go.
Use this in the buffer cache to free all the pages from a buffer,
resulting in a considerable speedup when throwing away pages from
the buffer cache.

Lots of work done with mlarkin and kettenis

ok kettinis@ deraadt@

show more ...

move the vm_page struct from being stored in RB macro trees to RBT functions

vm_page structs go into three trees, uvm_objtree, uvm_pmr_addr, and
uvm_pmr_size. all these have been moved to RBT code.

move the vm_page struct from being stored in RB macro trees to RBT functions

vm_page structs go into three trees, uvm_objtree, uvm_pmr_addr, and
uvm_pmr_size. all these have been moved to RBT code.

this should give us a decent chunk of code space back.

show more ...

Remove the unused loan_count field and the related uvm logic. Most of
the page loaning code is already in the Attic.

ok kettenis@, beck@

remove lock.h from uvm_extern.h. another holdover from the simpletonlock
era. fix uvm including c files to include lock.h or atomic.h as necessary.
ok deraadt

Prefer MADV_* over POSIX_MADV_* in kernel for consistency: the latter
doesn't have all the values and therefore can't be used everywhere.

ok deraadt@ kettenis@

Replace a plethora of historical protection options with just
PROT_NONE, PROT_READ, PROT_WRITE, and PROT_EXEC from mman.h.
PROT_MASK is introduced as the one true way of extracting those bits.
Remove

Replace a plethora of historical protection options with just
PROT_NONE, PROT_READ, PROT_WRITE, and PROT_EXEC from mman.h.
PROT_MASK is introduced as the one true way of extracting those bits.
Remove UVM_ADV_* wrapper, using the standard names.
ok doug guenther kettenis

show more ...

remove uneeded proc.h includes
ok mpi@ kspillner@

Fix some potential integer overflows caused by converting a page number into
an offset/size/address by shifting by PAGE_SHIFT. Make uvm_objwrire/unwire
use voff_t instead of off_t. The former is th

Fix some potential integer overflows caused by converting a page number into
an offset/size/address by shifting by PAGE_SHIFT. Make uvm_objwrire/unwire
use voff_t instead of off_t. The former is the right type here even if it is
equivalent to the latter.

Inspired by a somewhat similar changes in Bitrig.

ok deraadt@, guenther@

show more ...

remove simple_locks from uvm code. ok beck deraadt

Unbreak ramdisks.

I forgot that uvm_object.c wasn't build if SMALL_KERNEL. Fix this by building
the file unconditionally and only building the less used functions when
SMALL_KERNEL is not defined.

Unbreak ramdisks.

I forgot that uvm_object.c wasn't build if SMALL_KERNEL. Fix this by building
the file unconditionally and only building the less used functions when
SMALL_KERNEL is not defined.

unbreaks ramdisk build. ok jsg@

show more ...

Right now, if anything internal changes with a uvm object, diverse
places in the tree need to be touched to update the object
initialisation with respect to that.

So, make a function (uvm_initobj) t

Right now, if anything internal changes with a uvm object, diverse
places in the tree need to be touched to update the object
initialisation with respect to that.

So, make a function (uvm_initobj) that takes the refcount, object and
pager ops and does this initialisation for us. This should save on
maintainance in the future.

looked good to fgs@. Tedu complained about the British spelling but OKed
it anyway.

show more ...

the atomic primitives are still impossible to get at without using proc.h
(because it pulls in so much of the world) so include it for now, but
mark it XXX
ok tedu

something hairy happened with cvs so this file didn't get added.

re-add uvm_objwire and uvm_objunwire.

"you may commit that" kettenis@
original diff oked by ariane@ and art@

Add uvm_objwire() and uvm_objunwire(), I need this for some of my crazy
shitz.

Code mostly from netbsd with some tweaks from me. The loaning case is
specifcically if 0ed out right now because we do

Add uvm_objwire() and uvm_objunwire(), I need this for some of my crazy
shitz.

Code mostly from netbsd with some tweaks from me. The loaning case is
specifcically if 0ed out right now because we do not do loaning.

ok ariane@, art@.

show more ...