kernel - per-thread fd cache, p_fd lock bypass

* Implement a per-thread (fd,fp) cache. Cache hits can keep fp's
in a held state (avoiding the need to fhold()/fdrop() the ref count),
and bypasse

kernel - per-thread fd cache, p_fd lock bypass

* Implement a per-thread (fd,fp) cache. Cache hits can keep fp's
in a held state (avoiding the need to fhold()/fdrop() the ref count),
and bypasses the p_fd spinlock. This allows the file pointer structure
to generally be shared across cpu caches.

* Can cache up to four descriptors in each thread, LRU. This is the common
case. Highly threaded programs tend to focus work on a distinct
file descriptors in each thread.

* One file descriptor can be cached in up to four threads. This is
a significant limitation, though relatively uncommon. On a cache miss
the code drops into the normal shared p_fd spinlock lookup.

show more ...

devfs(9): Rename DEVFS_DECLARE_CLONE_BITMAP to DEVFS_DEFINE_CLONE_BITMAP.

Also, add DEVFS_DECLARE_CLONE_BITMAP() for extern declarations, analogous
to MALLOC_DEFINE() and MALLOC_DECLARE().

In the s

devfs(9): Rename DEVFS_DECLARE_CLONE_BITMAP to DEVFS_DEFINE_CLONE_BITMAP.

Also, add DEVFS_DECLARE_CLONE_BITMAP() for extern declarations, analogous
to MALLOC_DEFINE() and MALLOC_DECLARE().

In the sound code, replace some externs with DEVFS_DECLARE_CLONE_BITMAP()
and remove one unneeded extern.

show more ...

Initial import of binutils 2.22 on the new vendor branch

Future versions of binutils will also reside on this branch rather
than continuing to create new binutils branches for each new version.

DEVFS - move devfs headers to sys/sys; fix consumers

* Move devfs.h and devfs_rules.h from sys/vfs/devfs/ to sys/sys/

* Fix all consumers of the devfs headers to use the new path

DEVFS - add make_autoclone_dev to take of clone initializing for devs

by creating the device with the right miinor, initing the clone bitmap
and registering the clone handler. Also fix up all users

DEVFS - add make_autoclone_dev to take of clone initializing for devs

by creating the device with the right miinor, initing the clone bitmap
and registering the clone handler. Also fix up all users of cloning
to use this new API.

show more ...

DEVFS - rollup - all kernel devices

* Make changes needed to kernel devices to use devfs.

* Also pre-generate some devices (usually 4) to support system utilities
which do not yet deal with the a

DEVFS - rollup - all kernel devices

* Make changes needed to kernel devices to use devfs.

* Also pre-generate some devices (usually 4) to support system utilities
which do not yet deal with the auto-cloning device support.

* Adjust the spec_vnops for various filesystems to vector to dummy
code for read and write, for VBLK/VCHR nodes in old filesystems
which are no longer supported.

Submitted-by: Alex Hornung <ahornung@gmail.com>

show more ...

DEVFS - remove dev_ops_add(), dev_ops_get(), and get_dev()

DEVFS - Bring in Alex's GSOC kernel adjustments.

This is a rollup commit bringing in Alex Hornung's GSOC adjustments
to the main kernel codebase for DEVFS.

Submitted-by: Alex Hornung <ahornung@gmai

DEVFS - Bring in Alex's GSOC kernel adjustments.

This is a rollup commit bringing in Alex Hornung's GSOC adjustments
to the main kernel codebase for DEVFS.

Submitted-by: Alex Hornung <ahornung@gmail.com>

show more ...

Fix the way <sys/ioccom.h> is included throughout our tree.

The original intention was to include it only in header files which
define ioctl codes and not in .c or .h files which include headers
tha

Fix the way <sys/ioccom.h> is included throughout our tree.

The original intention was to include it only in header files which
define ioctl codes and not in .c or .h files which include headers
that define ioctl codes.

Adjust the tree to follow this idea.

Pointed-out-by: Guy Harris <guy@alum.mit.edu>
Dragonfly-bug: <http://bugs.dragonflybsd.org/issue1334>

show more ...

Replace NOCDEV with NULL. NOCDEV was ((void *)-1) and as inherited
from *BSD a long time ago due to the device pointer / device number
duality. Now that the pointer and device number has been separ

Replace NOCDEV with NULL. NOCDEV was ((void *)-1) and as inherited
from *BSD a long time ago due to the device pointer / device number
duality. Now that the pointer and device number has been separated, we
can just use NULL to indicate no-pointer.

Replace si_refs with si_sysref. Use SYSREF for ref-count cdev_t. Enable
cdev_t reclamation on deletion.

show more ...

Rename printf -> kprintf in sys/ and add some defines where necessary
(files which are used in userland, too).

Change the kernel dev_t, representing a pointer to a specinfo structure,
to cdev_t. Change struct specinfo to struct cdev. The name 'cdev' was taken
from FreeBSD. Remove the dev_t shim for the ker

Change the kernel dev_t, representing a pointer to a specinfo structure,
to cdev_t. Change struct specinfo to struct cdev. The name 'cdev' was taken
from FreeBSD. Remove the dev_t shim for the kernel.

This commit generally removes the overloading of 'dev_t' between userland and
the kernel.

Also fix a bug in libkvm where a kernel dev_t (now cdev_t) was not being
properly converted to a userland dev_t.

show more ...

Rename the kernel NODEV to NOCDEV to avoid conflicts with the userland NODEV.

Rename malloc->kmalloc, free->kfree, and realloc->krealloc. Pass 1

MASSIVE reorganization of the device operations vector. Change cdevsw
to dev_ops. dev_ops is a syslink-compatible operations vector structure
similar to the vop_ops structure used by vnodes.

Remov

MASSIVE reorganization of the device operations vector. Change cdevsw
to dev_ops. dev_ops is a syslink-compatible operations vector structure
similar to the vop_ops structure used by vnodes.

Remove a huge number of instances where a thread pointer is still being
passed as an argument to various device ops and other related routines.
The device OPEN and IOCTL calls now take a ucred instead of a thread pointer,
and the CLOSE call no longer takes a thread pointer.

show more ...

Convert most manual accesses to filedesc->fd_files[] into the appropriate
holdfp() call. Fix a number of places where ops were being executed
on the file pointer without holding a private reference

Convert most manual accesses to filedesc->fd_files[] into the appropriate
holdfp() call. Fix a number of places where ops were being executed
on the file pointer without holding a private reference to it (mainly
fo_ioctl(), revoke(), and lseek()).

Create procedures in kern_descrip.c to set and clear descriptor flags
and to handle the bootstrap filedesc for proc0. Replace manual code
elsewhere with calls to the new procedures.

Move getvnode() to kern_descrip.c. Remove nsmb_getfp(). Use holdfp()
instead.

show more ...

File descriptor cleanup stage 2, remove the separate arrays for file
pointers, fileflags, and allocation counts and replace the mess with a
single structural array. Also revamp the code that checks

File descriptor cleanup stage 2, remove the separate arrays for file
pointers, fileflags, and allocation counts and replace the mess with a
single structural array. Also revamp the code that checks whether the
file descriptor array is built-in or allocated.

Note that the removed malloc's were doing something weird, allocating
'nf * OFILESIZE + 1' bytes instead of 'nf * OFILESIZE' bytes. I could
not find any reason at all why it was doing that. It's gone now anyway.

show more ...

spl->critical section conversion.

Device layer rollup commit.

* cdevsw_add() is now required. cdevsw_add() and cdevsw_remove() may specify
a mask/match indicating the range of supported minor numbers. Multiple
cdevsw_add()'s u

Device layer rollup commit.

* cdevsw_add() is now required. cdevsw_add() and cdevsw_remove() may specify
a mask/match indicating the range of supported minor numbers. Multiple
cdevsw_add()'s using the same major number, but distinctly different
ranges, may be issued. All devices that failed to call cdevsw_add() before
now do.

* cdevsw_remove() now automatically marks all devices within its supported
range as being destroyed.

* vnode->v_rdev is no longer resolved when the vnode is created. Instead,
only v_udev (a newly added field) is resolved. v_rdev is resolved when
the vnode is opened and cleared on the last close.

* A great deal of code was making rather dubious assumptions with regards
to the validity of devices associated with vnodes, primarily due to
the persistence of a device structure due to being indexed by (major, minor)
instead of by (cdevsw, major, minor). In particular, if you run a program
which connects to a USB device and then you pull the USB device and plug
it back in, the vnode subsystem will continue to believe that the device
is open when, in fact, it isn't (because it was destroyed and recreated).

In particular, note that all the VFS mount procedures now check devices
via v_udev instead of v_rdev prior to calling VOP_OPEN(), since v_rdev
is NULL prior to the first open.

* The disk layer's device interaction has been rewritten. The disk layer
(i.e. the slice and disklabel management layer) no longer overloads
its data onto the device structure representing the underlying physical
disk. Instead, the disk layer uses the new cdevsw_add() functionality
to register its own cdevsw using the underlying device's major number,
and simply does NOT register the underlying device's cdevsw. No
confusion is created because the device hash is now based on
(cdevsw,major,minor) rather then (major,minor).

NOTE: This also means that underlying raw disk devices may use the entire
device minor number instead of having to reserve the bits used by the disk
layer, and also means that can we (theoretically) stack a fully
disklabel-supported 'disk' on top of any block device.

* The new reference counting scheme prevents this by associating a device
with a cdevsw and disconnecting the device from its cdevsw when the cdevsw
is removed. Additionally, all udev2dev() lookups run through the cdevsw
mask/match and only successfully find devices still associated with an
active cdevsw.

* Major work on MFS: MFS no longer shortcuts vnode and device creation. It
now creates a real vnode and a real device and implements real open and
close VOPs. Additionally, due to the disk layer changes, MFS is no longer
limited to 255 mounts. The new limit is 16 million. Since MFS creates a
real device node, mount_mfs will now create a real /dev/mfs<PID> device
that can be read from userland (e.g. so you can dump an MFS filesystem).

* BUF AND DEVICE STRATEGY changes. The struct buf contains a b_dev field.
In order to properly handle stacked devices we now require that the b_dev
field be initialized before the device strategy routine is called. This
required some additional work in various VFS implementations. To enforce
this requirement, biodone() now sets b_dev to NODEV. The new disk layer
will adjust b_dev before forwarding a request to the actual physical
device.

* A bug in the ISO CD boot sequence which resulted in a panic has been fixed.

Testing by: lots of people, but David Rhodus found the most aggregious bugs.

show more ...

device switch 1/many: Remove d_autoq, add d_clone (where d_autoq was).

d_autoq was used to allow the device port dispatch to mix old-style synchronous
calls with new style messaging calls within a p

device switch 1/many: Remove d_autoq, add d_clone (where d_autoq was).

d_autoq was used to allow the device port dispatch to mix old-style synchronous
calls with new style messaging calls within a particular device. It was never
used for that purpose.

d_clone will be more fully implemented as work continues. We are going to
install d_port in the dev_t (struct specinfo) structure itself and d_clone
will be needed to allow devices to 'revector' the port on a minor-number
by minor-number basis, in particular allowing minor numbers to be directly
dispatched to distinct threads. This is something we will be needing later
on.

show more ...

Get rid of the obsolete SMP checks in SMBFS.

kernel tree reorganization stage 1: Major cvs repository work (not logged as
commits) plus a major reworking of the #include's to accomodate the
relocations.

* CVS repository files manually move

kernel tree reorganization stage 1: Major cvs repository work (not logged as
commits) plus a major reworking of the #include's to accomodate the
relocations.

* CVS repository files manually moved. Old directories left intact
and empty (temporary).

* Reorganize all filesystems into vfs/, most devices into dev/,
sub-divide devices by function.

* Begin to move device-specific architecture files to the device
subdirs rather then throwing them all into, e.g. i386/include

* Reorganize files related to system busses, placing the related code
in a new bus/ directory. Also move cam to bus/cam though this may
not have been the best idea in retrospect.

* Reorganize emulation code and place it in a new emulation/ directory.

* Remove the -I- compiler option in order to allow #include file
localization, rename all config generated X.h files to use_X.h to
clean up the conflicts.

* Remove /usr/src/include (or /usr/include) dependancies during the
kernel build, beyond what is normally needed to compile helper
programs.

* Make config create 'machine' softlinks for architecture specific
directories outside of the standard <arch>/include.

* Bump the config rev.

WARNING! after this commit /usr/include and /usr/src/sys/compile/*
should be regenerated from scratch.

show more ...

DEV messaging stage 1/4: Rearrange struct cdevsw and add a message port
and auto-queueing mask. The mask will tell us which message functions
can be safely queued to another thread and which still n

DEV messaging stage 1/4: Rearrange struct cdevsw and add a message port
and auto-queueing mask. The mask will tell us which message functions
can be safely queued to another thread and which still need to run in the
context of the caller. Primary configuration fields (name, cmaj, flags,
port, autoq mask) are now at the head of the structure. Function vectors,
which may eventually go away, are at the end. The port and autoq fields
are non-functional in this stage.

The old BDEV device major number support has also been removed from cdevsw,
and code has been added to translate the bootdev passed from the boot code
(the boot code has always passed the now defunct block device major numbers
and we obviously need to keep that compatibility intact).

show more ...

proc->thread stage 4: rework the VFS and DEVICE subsystems to take thread
pointers instead of process pointers as arguments, similar to what FreeBSD-5
did. Note however that ultimately both APIs are

proc->thread stage 4: rework the VFS and DEVICE subsystems to take thread
pointers instead of process pointers as arguments, similar to what FreeBSD-5
did. Note however that ultimately both APIs are going to be message-passing
which means the current thread context will not be useable for creds and
descriptor access.

show more ...

proc->thread stage 2: MAJOR revamping of system calls, ucred, jail API,
and some work on the low level device interface (proc arg -> thread arg).
As -current did, I have removed p_cred and incorporat

proc->thread stage 2: MAJOR revamping of system calls, ucred, jail API,
and some work on the low level device interface (proc arg -> thread arg).
As -current did, I have removed p_cred and incorporated its functions
into p_ucred. p_prison has also been moved into p_ucred and adjusted
accordingly. The jail interface tests now uses ucreds rather then processes.

The syscall(p,uap) interface has been changed to just (uap). This is inclusive
of the emulation code. It makes little sense to pass a proc pointer around
which confuses the MP readability of the code, because most system call code
will only work with the current process anyway. Note that eventually
*ALL* syscall emulation code will be moved to a kernel-protected userland
layer because it really makes no sense whatsoever to implement these
emulations in the kernel.

suser() now takes no arguments and only operates with the current process.
The process argument has been removed from suser_xxx() so it now just takes
a ucred and flags.

The sysctl interface was adjusted somewhat.

show more ...