Use reclaim2 to fix reclaim/lookup race conditions

The PUFFS reclaim operation had a race condition with lookups: we could
be asked to lookup a node, then to reclaim it before lookup completion.
At

Use reclaim2 to fix reclaim/lookup race conditions

The PUFFS reclaim operation had a race condition with lookups: we could
be asked to lookup a node, then to reclaim it before lookup completion.
At lookup completion, we would then create a leaked node.

Enter the PUFFS reclaim2 operation, which features a nlookup argument.
That let us count how many lookups are pending and avoid the above
described scenario. It also makes the codes simplier.

show more ...

- Fix same vnodes associated with multiple cookies
The scheme used to retreive known nodes on lookup was flawed, as it only
used parent and name. This produced a different cookie for the same file
if

- Fix same vnodes associated with multiple cookies
The scheme used to retreive known nodes on lookup was flawed, as it only
used parent and name. This produced a different cookie for the same file
if it was renamed, when looking up ../ or when dealing with multiple files
associated with the same name through link(2).

We therefore abandon the use of node name and introduce hashed lists of
inodes. This causes a huge rewrite of reclaim code, which do not attempt
to keep parents allocated until all their children are reclaimed

- Fix race conditions in reclaim
There are a few situations where we issue multiple FUSE operations for
a PUFFS operation. On reclaim, we therefore have to wait for all FUSE
operation to complete, not just the current exchanges. We do this by
introducing node reference count with node_ref() and node_rele().

- Detect data loss caused by FAF
VOP_PUTPAGES causes FAF writes where the kernel does not check the
operation result. At least issue a warning on error.

- Enjoy FAF shortcut on setattr
No need to wait for the result if the kernel does not want it. There is
however an exception for setattr that touch the size, we need to wait
for completion because we have other operations queued for after the
resize.

- Fix fchmod() on write-open file
fchmod() on a node open with write privilege will send setattr with both mode and size set. This confuses some FUSE filesystem. Therefore we send two FUSE operations, one for mode, and one for size.

- Remove node TTL handling for netbsd-5 for simplicity sake. The code
still builds on netbsd-5 but does not have the node TTL feature anymore.
It works fine with kernel support on netbsd-6.

show more ...

Use C89 function definition

Fix 32-bit build

Improve FUSE trace facility

Redo previous; remove all the casts I added, and use PRI* macros instead.
(by popular demand - makes sense, too)

Cast time_t to intmax_t for printf purposes, and format with %j. Fixes
build on amd64 and probably i386 as well.

Add a FUSE trace facility, with statistics collection. This should help
tracking bugs and performance issues

- Correctly handle rename whith overwritten destination
- Keep track of file name to avoid lookups when we can. This makes sure we
do not have two cookies for the same inode, a situation that cause

- Correctly handle rename whith overwritten destination
- Keep track of file name to avoid lookups when we can. This makes sure we
do not have two cookies for the same inode, a situation that cause wreak
havoc when we come to remove or rename a node.
- Do not use PUFFS_FLAG_BUILDPATH at all, since we now track file names
- In open, queue requests after checking for access, as there is no merit
to queue a will-be-denied request while we can deny it immediatly
- request reclaim of removed nodes at inactive stage

show more ...

= Open files =
- Restore open on our own in fsycn and readdir, as the node may not already
be open, and FUSE really wants it to be. No need to close immediatly, it
can be done at inactive time.

= Wr

= Open files =
- Restore open on our own in fsycn and readdir, as the node may not already
be open, and FUSE really wants it to be. No need to close immediatly, it
can be done at inactive time.

= Write operations =
- fix a nasty bug that corrupted files on write (written added twice)
- Keep track of file size in order to honour PUFFS_IO_APPEND

= many fixes in rename =
- handler overwritten nodes correctly
- wait for all operations on the node to drain before doing rename, as
filesystems may not cope with operations on a moving file.
- setback PUFFS_SETBACK_INACT_N1 cannot be used from rename, we therefore
miss the inactive time for an overwritten node. This bounds us to give up
PUFFS_KFLAG_IAONDEMAND.

= Removed files =
- forbid most operations on a removed node, return ENOENT
- setback PUFFS_SETBACK_NOREF_N1 at inactive stage to cause removed
file reclaim

= Misc =
- Update outdated ARGSUSED for lint
- Fix a memory leak (puffs_pn_remove instead of puffs_pn_put)
- Do not use PUFFS_FLAG_BUILDPATH except for debug output. It makes the
lookup code much simplier.

show more ...

== file close operations ==
- use PUFFS_KFLAG_WTCACHE to puffs_init so that all writes are
immediatly send to the filesystem, and we do not have anymore write
after inactive. As a consequence, we can

== file close operations ==
- use PUFFS_KFLAG_WTCACHE to puffs_init so that all writes are
immediatly send to the filesystem, and we do not have anymore write
after inactive. As a consequence, we can close files at inactive
stage, and there is not any concern left with files opened at
create time. We also do not have anymore to open ourselves in readdir and
fsync.

- Fsync on close (inactive stage). That makes sure we will not need to
do these operations once the file is closed (FUSE want an open file).
short sircuit the request that come after the close, bu not fsinc'ing
closed files,

- Use PUFFS_KFLAG_IAONDEMAND to get less inactive calls

== Removed nodes ==
- more ENOENT retunred for operations on removed node (but there
are probably some still missing): getattr, ooen, setattr, fsync

- set PND_REMOVE before sending the UNLINK/RMDIR operations so that we avoid
races during UNLINK completion. Also set PND_REMOVED on node we overwirte
in rename

== Filehandle fixes ==
- queue open operation to avoid getting two fh for one file

- set FH in getattr, if the file is open

- Just requires a read FH for fsyncdir, as we always opendir in read
mode. Ok, this is misleading :-)

== Misc ==
- do not set FUSE_FATTR_ATIME_NOW in setattr, as we provide the time

- short circuit nilpotent operations in setattr

- add a filename diagnostic flag to dump file names

show more ...

- Use SOCK_DGRAM instead of SOCK_STREAM, as the filesystem seems to
assume datagram semantics: when using SOCK_STREAM, if perfused sends
frames faster than the filesystem consumes them, it will grab

- Use SOCK_DGRAM instead of SOCK_STREAM, as the filesystem seems to
assume datagram semantics: when using SOCK_STREAM, if perfused sends
frames faster than the filesystem consumes them, it will grab multiple
frames at once and discard anything beyond the first one. For now the
code can work both with SOCK_DGRAM and SOCK_STREAM, but SOCK_STREAM
support will probably have to be removed for the sake of readability.

- Remeber to sync parent directories when moving a node

- In debug output, display the requeue type (readdir, write, etc...)

show more ...

libperfuse(3) is a PUFFS relay to FUSE. In order to use it,
FUSE filesystem must be patched to #include <perfuse.h> in the source
files that open /dev/fuse and perform the mount(2) system call. The
F

libperfuse(3) is a PUFFS relay to FUSE. In order to use it,
FUSE filesystem must be patched to #include <perfuse.h> in the source
files that open /dev/fuse and perform the mount(2) system call. The
FUSE filesystem must be linked with -lperfuse.

libperfuse(3) implements the FUSE kernel interface, on which libfuse or
any FUSE filesystem that opens /dev/fuse directly can be used.

For now, an external daemon called perfused(8) is used. This may change
in the future.

show more ...