File descriptor changes, discussed on tech-kern:

- Redo reference counting to be sane. LWPs accessing files take a short
term reference on the local file descriptor. This is the most common
ca

File descriptor changes, discussed on tech-kern:

- Redo reference counting to be sane. LWPs accessing files take a short
term reference on the local file descriptor. This is the most common
case. While a file is in a process descriptor table, a reference is
held to the file. The file reference count only changes during control
operations like open() or close(). Code that comes at files from an
unusual direction (i.e. foreign to the process) like procfs or sysctl
takes a reference on the file (f_count), and not on a descriptor.

- Remove knowledge of reference counting and locking from most code that
deals with files.

- Make the usual case of file descriptor lookup lockless.

- Make kqueue MP and MT safe. PR kern/38098, PR kern/38137.

- Fix numerous file handling bugs, and bugs in the descriptor code that
affected multithreaded processes.

- Split descriptor system calls out into sys_descrip.c.

- A few stylistic changes: KNF, remove unused casts now that caddr_t is
gone. Replace dumb gotos with loop control in a few places.

- Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless
the routine is likely to be inlined. Most of the time it's about the
current process.

show more ...

Move the prototype for do_posix_fadvise() somewhere useful.

Add fgetdummy/fputdummy: allocate and free dummy 'struct file' entries
to be used when traversing filehead.

- add an elf aux vector entry for implementing $ORIGIN.
- the code to convert from a vnode to a path is commented out now until
a better solution is implemented. Only absolute paths work for now

- add an elf aux vector entry for implementing $ORIGIN.
- the code to convert from a vnode to a path is commented out now until
a better solution is implemented. Only absolute paths work for now
(which is most of the cases).

requested by core

show more ...

Use atomics to adjust cwdi_refcnt.

Merge file descriptor locking, cwdi locking and cross-call changes
from the vmlocking branch.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

Split the fcntl locking code out from its copyin/out.
Use to avoid all the stackgap stuff in compat code.

Change the way that emulations locate files within the emulation root to
avoid having to allocate space in the 'stackgap'
- which is very LWP unfriendly.
The additional code for non-emulation nam

Change the way that emulations locate files within the emulation root to
avoid having to allocate space in the 'stackgap'
- which is very LWP unfriendly.
The additional code for non-emulation namei() is trivial, the reduction for
the emulations is massive.
The vnode for a processes emulation root is saved in the cwdi structure
during process exec.
If the emulation root the TRYEMULROOT flag are set, namei() will do an initial
search for absolute pathnames in the emulation root, if that fails it will
retry from the normal root.
".." at the emulation root will always go to the real root, even in the middle
of paths and when expanding symlinks.
Absolute symlinks found using absolute paths in the emulation root will be
relative to the emulation root (so /usr/lib/xxx.so -> /lib/xxx.so links
inside the emulation root don't need changing).
If the root of the emulation would be returned (for an emulation lookup), then
the real root is returned instead (matching the behaviour of emul_lookup,
but being a cheap comparison here) so that programs that scan "../.."
looking for the root dircetory don't loop forever.
The target for symbolic links is no longer mangled (it used to get the
CHECK_ALT_xxx() treatment, so could get /emul/xxx prepended).
CHECK_ALT_xxx() are no more. Most of the change is deleting them, and adding
TRYEMULROOT to the flags to NDINIT().
A lot of the emulation system call stubs could now be deleted.

show more ...

Split the work for sys_stat, sys_lstat, sys_fstat and sys_fhstat out into
separate functions that don't do the copyout.
This allows all the compat_xxx versions to convert the 'struct stat' to
the cor

Split the work for sys_stat, sys_lstat, sys_fstat and sys_fhstat out into
separate functions that don't do the copyout.
This allows all the compat_xxx versions to convert the 'struct stat' to
the correct format without using the 'stackgap'.
The stackgap isn't at all LWP friendly, and needs to be removed from
any compat functions that might involve threads (inc. clone()).
The code is still binary compatible with existing LKMs.

show more ...

Use the LWP cached credentials where sane.

remove the no longer useful fdavail(), as proposed and (thankfully) not
discussed on tech-kern

merge ktrace-lwp.

#include sys/lock.h for struct simplelock definition.

Addresses PR pkg/25772.

Implement mutexes for file descriptor and current working directory access.
Fix a potential race condition when reallocating storage for file descriptors
(even for non-SMP kernels).
Add missing locks

Implement mutexes for file descriptor and current working directory access.
Fix a potential race condition when reallocating storage for file descriptors
(even for non-SMP kernels).
Add missing locks for `struct file' ref count updates.

show more ...

use a two-level bitmap as suggested by mogul and banga for fdalloc;
approved thorpej@

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

no variable in prototype

Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various fu

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

show more ...

move declaration of getcwd_common and constant to the header file.

merge kqueue branch into -current

kqueue provides a stateful and efficient event notification framework
currently supported events include socket, file, directory, fifo,
pipe, tty and device changes

merge kqueue branch into -current

kqueue provides a stateful and efficient event notification framework
currently supported events include socket, file, directory, fifo,
pipe, tty and device changes, and monitoring of processes and signals

kqueue is supported by all writable filesystems in NetBSD tree
(with exception of Coda) and all device drivers supporting poll(2)

based on work done by Jonathan Lemon for FreeBSD
initial NetBSD port done by Luke Mewburn and Jason Thorpe

show more ...

From OpenBSD, via FreeBSD: If a set{u,g}id binary is invoked with fd < 3
closed, open those fds to /dev/null.

XXX: This needs to be fixed in a better way. The kernel should not need to
know about /d

From OpenBSD, via FreeBSD: If a set{u,g}id binary is invoked with fd < 3
closed, open those fds to /dev/null.

XXX: This needs to be fixed in a better way. The kernel should not need to
know about /dev/null or special case 0, 1, 2.

show more ...

Fix a partial construction problem that can cause race conditions
between creation of a file descriptor and close(2) when using kernel
assisted threads. What we do is stick descriptors in the table,

Fix a partial construction problem that can cause race conditions
between creation of a file descriptor and close(2) when using kernel
assisted threads. What we do is stick descriptors in the table, but
mark them as "larval". This causes essentially everything to treat
it as a non-existent descriptor, except for fdalloc(), which sees a
filled slot so that it won't (incorrectly) allocate it again. When
a descriptor is fully constructed, the code that has constructed it
marks it as "mature" (which actually clears the "larval" flag), and
things continue to work as normal.

While here, gather all the code that gets a descriptor from the table
into a fd_getfile() function, and call it, rather than having the
same (sometimes incorrect) code copied all over the place.

show more ...

Rework fdalloc() even further: split fdalloc() into fdalloc() and
fdexpand(). The former will return ENOSPC if there is not space
in the current filedesc table. The latter performs the expansion
of

Rework fdalloc() even further: split fdalloc() into fdalloc() and
fdexpand(). The former will return ENOSPC if there is not space
in the current filedesc table. The latter performs the expansion
of the filedesc table. This means that fdalloc() won't ever block,
and it gives callers an opportunity to clean up before the
potentially-blocking fdexpand() call.

Update all fdalloc() callers to deal with the need-to-fdexpand() case.

Rewrite unp_externalize() to use fdalloc() and fdexpand() in a
safe way, using an algorithm suggested by Bill Sommerfeld:
- Use a temporary array of integers to hold the new filedesc table
indexes. This allows us to repeat the loop if necessary.
- Loop through the array of file *'s, assigning them to filedesc table
slots. If fdalloc() indicates expansion is necessary, undo the
assignments we've done so far, expand, and retry the whole process.
- Once all file *'s have been assigned to slots, update the f_msgcount
and unp_rights counters.
- Right before we return, copy the temporary integer array to the message
buffer, and trim the length as before.
Note that once locking is added to the filedesc array, this entire
operation will be `atomic', in that the lock will be held while
file *'s are assigned to embryonic table slots, thus preventing anything
else from using them.

show more ...