| ef8d499e | 29-Sep-2016 |
David van Moolenbroek <david@minix3.org> |
Add lwip: a new lwIP-based TCP/IP service
This commit adds a new TCP/IP service to MINIX 3. As its core, the
service uses the lwIP TCP/IP stack for maintenance reasons. The
service aims to be comp
Add lwip: a new lwIP-based TCP/IP service
This commit adds a new TCP/IP service to MINIX 3. As its core, the
service uses the lwIP TCP/IP stack for maintenance reasons. The
service aims to be compatible with NetBSD userland, including its
low-level network management utilities. It also aims to support
modern features such as IPv6. In summary, the new LWIP service has
support for the following main features:
- TCP, UDP, RAW sockets with mostly standard BSD API semantics;
- IPv6 support: host mode (complete) and router mode (partial);
- most of the standard BSD API socket options (SO_);
- all of the standard BSD API message flags (MSG_);
- the most used protocol-specific socket and control options;
- a default loopback interface and the ability to create one more;
- configuration-free ethernet interfaces and driver tracking;
- queuing and multiple concurrent requests to each ethernet driver;
- standard ioctl(2)-based BSD interface management;
- radix tree backed, destination-based routing;
- routing sockets for standard BSD route reporting and management;
- multicast traffic and multicast group membership tracking;
- Berkeley Packet Filter (BPF) devices;
- standard and custom sysctl(7) nodes for many internals;
- a slab allocation based, hybrid static/dynamic memory pool model.
Many of its modules come with fairly elaborate comments that cover
many aspects of what is going on. The service is primarily a socket
driver built on top of the libsockdriver library, but for BPF devices
it is at the same time also a character driver.
Change-Id: Ib0c02736234b21143915e5fcc0fda8fe408f046f
show more ...
|
| 0f03189a | 30-Sep-2016 |
David van Moolenbroek <david@minix3.org> |
RMIB: add indirection support for sparse subtrees
Normally, each RMIB subtree consists of an array of nodes, indexed
by node identifier. In a sparsely filled subtree, most of the array
is empty and
RMIB: add indirection support for sparse subtrees
Normally, each RMIB subtree consists of an array of nodes, indexed
by node identifier. In a sparsely filled subtree, most of the array
is empty and just wasting memory. In that case, it may be beneficial
to have a level of indirection, with an intermediate array containing
pairs of node IDs and pointers to the actual nodes. This patch adds
support for such indirection.
For the use cases that inspired this patch, net.inet and net.inet6,
the indirection shaves off a little under 16KB of memory from the
TCP/IP service.
Change-Id: Ic68ca3fee1a0f2032f77eef6df42728f9b9400e8
show more ...
|
| 49bf8d5c | 14-Feb-2017 |
David van Moolenbroek <david@minix3.org> |
Retire MINIX tcpstat(1), udpstat(1)
This needs to be done before retiring inet itself, since these
utilities include headers from inet directly.
Also retire the now-obsolete paramvalue(3).
Change-
Retire MINIX tcpstat(1), udpstat(1)
This needs to be done before retiring inet itself, since these
utilities include headers from inet directly.
Also retire the now-obsolete paramvalue(3).
Change-Id: I9b27771190a6a32ee533b0c0d9d37f61a16ee36c
show more ...
|
| 27852ebe | 21-Feb-2016 |
David van Moolenbroek <david@minix3.org> |
UDS: full rewrite
This new implementation of the UDS service is built on top of the
libsockevent library. It thereby inherits all the advantages that
libsockevent brings. However, the fundamental
UDS: full rewrite
This new implementation of the UDS service is built on top of the
libsockevent library. It thereby inherits all the advantages that
libsockevent brings. However, the fundamental restructuring
required for that change also paved the way for resolution of a
number of other important open issues with the old UDS code. Most
importantly, the rewrite brings the behavior of the service much
closer to POSIX compliance and NetBSD compatibility. These are the
most important changes:
- due to the use of libsockevent, UDS now supports multiple suspending
calls per socket and a large number of standard socket flags and
options;
- socket address matching is now based on <device,inode> lookups
instead of canonized path names, and socket addresses are no longer
altered either due to canonization or at connect time;
- the socket state machine is now well defined, most importantly
resolving the erroneous reset-on-EOF semantics of the old UDS, but
also allowing socket reuse;
- sockets are now connected before being accepted instead of being
held in connecting state, unless the LOCAL_CONNWAIT option is set
on either the connecting or the listening socket;
- connect(2) on datagram sockets is now supported (needed by syslog),
and proper datagram socket disconnect notification is provided;
- the receive queue now supports segmentation, associating ancillary
data (in-flight file descriptors and credentials) with each segment
instead of being kept fully separately; this is a POSIX requirement
(and needed by tmux);
- as part of the segmentation support, the receive queue can now hold
as many packets as can fit, instead of one;
- in addition to the flags supported by libsockevent, the MSG_PEEK,
MSG_WAITALL, MSG_CMSG_CLOEXEC, MSG_TRUNC, and MSG_CTRUNC send and
receive flags are now supported;
- the SO_PASSCRED and SO_PEERCRED socket options are replaced by
LOCAL_CREDS and LOCAL_PEEREID respectively, now following NetBSD
semantics and allowing use of NetBSD libc's getpeereid(3);
- memory usage is reduced by about 250 KB due to centralized in-flight
file descriptor tracking, with a limit of OPEN_MAX total rather than
of OPEN_MAX per socket;
- memory usage is reduced by another ~50 KB due to removal of state
redundancy, despite the fact that socket path names may now be up to
253 bytes rather than the previous 104 bytes;
- compared to the old UDS, there is now very little direct indexing on
the static array of sockets, thus allowing dynamic allocation of
sockets more easily in the future;
- the UDS service now has RMIB support for the net.local sysctl tree,
implementing preliminary support for NetBSD netstat(1).
Change-Id: I4a9b6fe4aaeef0edf2547eee894e6c14403fcb32
show more ...
|
| 241ebcae | 30-Sep-2016 |
David van Moolenbroek <david@minix3.org> |
RMIB: expose full node path; improve restartability
A single function may be used to handle the implementation of more
than one node. In some cases, the behavior of that function may
depend on the
RMIB: expose full node path; improve restartability
A single function may be used to handle the implementation of more
than one node. In some cases, the behavior of that function may
depend on the path name used to reach the node. Therefore, provide
the full path name as part of the call information.
As a result, RMIB has to save the paths for each of its remote MIB
mount points. That in turn also allows it to autonomously remount its
mount points after a MIB service restart, thus bringing us a step
closer to proper recovery after a MIB crash without requiring the
service using RMIB to perform explicit steps. As before, the missing
ingredient is actual notification of MIB service restarts, and proper
support for *that* will likely require changes to the DS service.
Change-Id: Ic0c79931d6f3a76c2c998047f8b47350fd0fa5b0
show more ...
|
| bfa518c7 | 12-Jul-2016 |
David van Moolenbroek <david@minix3.org> |
PM/libsys: extend getepinfo, add getsockcred(3)
The service-only getepinfo(2) PM call returns information about a
given endpoint. This patch extends that call so that it returns
enough information
PM/libsys: extend getepinfo, add getsockcred(3)
The service-only getepinfo(2) PM call returns information about a
given endpoint. This patch extends that call so that it returns
enough information to allow correctly filling a sockcred structure.
A new getsockcred(3) function is added to libsys to fill an actual
sockcred structure with the obtained information. However, for the
caller's convenience, the groups list is kept separate.
Change-Id: I9f1a6d1a221c77eabaa3498ff4ec9a5fb922e4fd
show more ...
|
| 46271349 | 27-Jan-2016 |
David van Moolenbroek <david@minix3.org> |
libsys: retire getnucred in favor of getepinfo
The getnucred() function was used by UDS to obtain credentials of user
processes in a form used in the UDS API, namely the ucred structure.
Since the N
libsys: retire getnucred in favor of getepinfo
The getnucred() function was used by UDS to obtain credentials of user
processes in a form used in the UDS API, namely the ucred structure.
Since the NetBSD merge, this structure has changed drastically (aside
from being renamed to "uucred"), and it is no longer in UDS's best
interest to use this structure internally. Therefore, getnucred() is
no longer a useful API either, and instead we directly use the
previously private getepinfo() function to obtain credentials.
Change-Id: I80bc809de716ec0a9b7497cb109d2f2708a629d5
show more ...
|
| dd969671 | 27-Dec-2015 |
David van Moolenbroek <david@minix3.org> |
UDS: prepare for socket file creation in bind(2)
This patch prepares for moving of the creation of socket files on the
file system from the libc bind(2) stub into the UDS service. This
change is ne
UDS: prepare for socket file creation in bind(2)
This patch prepares for moving of the creation of socket files on the
file system from the libc bind(2) stub into the UDS service. This
change is necessary for the socket type agnostic libc implementation.
The change is not yet activated - the code that is not yet used is
enclosed in "#if NOT_YET" blocks. The activation needs to be atomic
with UDS's switch to libsockdriver; otherwise, user applications may
break.
As part of the change, various UDS bind(2) semantics are changed to
match the POSIX standard and other operating systems. In
implementation terms, the service-only VFS API checkperms(2) is
renamed to socketpath(2), and extended with a new subcall which
creates a new socket file. An extension to test56 checks the new
bind(2) semantics of UDS, although most new tests are still disabled
until activation as well.
Finally, as further preparation for a more structural redesign of the
UDS service, also return the <device,inode> number pair for the
created or checked file name, and make returning the canonized path
name optional.
Change-Id: I892d04b3301d4b911bdc571632ddde65fb747a8a
show more ...
|
| 4c27a833 | 21-Jul-2016 |
David van Moolenbroek <david@minix3.org> |
Add libsockevent: a socket event dispatching library
This library provides an event-based abstraction model and dispatching
facility for socket drivers. Its main goal is to eliminate any and
all ne
Add libsockevent: a socket event dispatching library
This library provides an event-based abstraction model and dispatching
facility for socket drivers. Its main goal is to eliminate any and
all need for socket drivers to keep track of pending socket calls.
Additionally, this library takes over responsibility of a number of
other tasks that would otherwise be duplicated between socket drivers,
but in such a way that individual socket drivers retain a large degree
of freedom in terms of API behavior. The library's main features are:
- suspension, resumption, and cancellation of socket calls;
- an abstraction layer for select(2);
- state tracking of shutdown(2);
- pending (asynchronous) errors and the SO_ERROR socket option;
- listening-socket tracking and the SO_ACCEPTCONN socket option;
- generation of SIGPIPE signals; SO_NOSIGPIPE, MSG_NOSIGNAL;
- send and receive low-watermark tracking, SO_SNDLOWAT, SO_RCVLOWAT;
- send and receive timeout support and SO_SNDTIMEO, SO_RCVTIMEO;
- an abstraction layer for the SO_LINGER socket option;
- tracking of various on/off socket options as well as SO_TYPE;
- a range of pre-checks on socket calls that are required POSIX.
In order to track per-socket state, the library manages an opaque
"sock" object for each socket. The allocation of such objects is left
entirely to the socket driver. Each sock object has an associated
callback table for calls from libsockevent to the socket driver. The
socket driver can raise events on the sock object in order to flag
that any previously suspended operations of a particular type should
be resumed. The library may defer processing such raised events if
immediate processing could interfere with internal consistency.
The sockevent library is layered on top of libsockdriver, and should
be used by all socket driver implementations if at all possible.
Change-Id: I3eb2c80602a63ef13035f646473360293607ab76
show more ...
|
| 85723df0 | 21-Feb-2016 |
David van Moolenbroek <david@minix3.org> |
Add libsockdriver: a library for socket drivers
This library provides abstractions for socket drivers, and should be
used as the basis for all socket driver implementations. It provides
the followi
Add libsockdriver: a library for socket drivers
This library provides abstractions for socket drivers, and should be
used as the basis for all socket driver implementations. It provides
the following functionality:
- a function call table abstraction, hiding the details of the
socket driver protocol with simple parameters and presenting the
socket driver with callback functions very similar to the BSD
socket API calls made from userland;
- abstracting data structures and helper functions for suspending
and resuming blocking calls;
- abstracting data structures and helper functions for copying data
from and to the caller.
Overall, the library is similar to lib{block,char,fs,input,net}driver
in concept. Some of the abstractions provided here should in fact be
applied to libchardriver as well. As always, for the case that the
provided message loop is too restrictive, a set of more low-level
message processing functions is provided.
Change-Id: I79ec215f5e195c3b0197e223636f987d3755fb13
show more ...
|
| 45443f35 | 20-Jul-2016 |
David van Moolenbroek <david@minix3.org> |
VFS: support close-on-exec flag for copyfd(2)
The flag is supported only when copying out file descriptors (i.e.
COPYFD_TO). It will be used by UDS to support MSG_CMSG_CLOEXEC.
Change-Id: I46bfd04
VFS: support close-on-exec flag for copyfd(2)
The flag is supported only when copying out file descriptors (i.e.
COPYFD_TO). It will be used by UDS to support MSG_CMSG_CLOEXEC.
Change-Id: I46bfd04b5f28e22ec48938e43e42f78d3931220d
show more ...
|
| 491d647a | 25-Jul-2016 |
David van Moolenbroek <david@minix3.org> |
VFS: support for suspending close(2) for sockets
This change effectively adds the VFS side of support for the SO_LINGER
socket option, by allowing file descriptor close operations to be
suspended (a
VFS: support for suspending close(2) for sockets
This change effectively adds the VFS side of support for the SO_LINGER
socket option, by allowing file descriptor close operations to be
suspended (and later resumed) by socket drivers. Currently, support
is limited to the close(2) system call--in all other cases where file
descriptors are closed (dup2, close-on-exec, process exit..), the
close operation still completes instantly. As a general policy, the
close(2) return value will always indicate that the file descriptor
has been closed: either 0, or -1 with errno set to EINPROGRESS. The
latter error may be thrown only when a suspended close is interrupted
by a signal.
As necessary for UDS, this change also introduces a closenb(2) system
call extension, allowing the caller to bypass blocking SO_LINGER close
behavior. This extension allows UDS to avoid blocking on closing the
last reference to an in-flight file descriptor, in an atomic fashion.
The extension is currently part of libsys, but there is no reason why
userland would not be allowed to make this call, so it is deliberately
not protected from use by userland.
Change-Id: Iec77d6665232110346180017fc1300b1614910b7
show more ...
|
| e3b8d4bb | 21-Feb-2016 |
David van Moolenbroek <david@minix3.org> |
VFS: add BSD socket API, socket driver support
This patch adds the implementation of the BSD socket system calls
which have been introduced in an earlier patch. At the same time, it
adds support fo
VFS: add BSD socket API, socket driver support
This patch adds the implementation of the BSD socket system calls
which have been introduced in an earlier patch. At the same time, it
adds support for communication with socket drivers, using a new
"socket device" (SDEV_) protocol. These two parts, implemented in
socket.c and sdev.c respectively, form the upper and lower halves of
the new BSD socket support in VFS. New mapping functionality for
socket domains and drivers is added as well, implemented in smap.c.
The rest of the changes mainly facilitate the separation of character
and socket driver calls, and do not make any fundamental alterations.
For example, while this patch changes VFS's select.c rather heavily,
the new select logic for socket drivers is the exact same as for
character drivers; the changes mainly separate the driver type
specific parts from the generic select logic further than before.
Change-Id: I2f13084dd3c8d3a68bfc69da0621120c8291f707
show more ...
|
| 181fb1b2 | 21-Feb-2016 |
David van Moolenbroek <david@minix3.org> |
RS: add infrastructure for mapping socket drivers
This patch introduces the first piece of support for the concept of
"socket drivers": services that implement one or more socket protocol
families.
RS: add infrastructure for mapping socket drivers
This patch introduces the first piece of support for the concept of
"socket drivers": services that implement one or more socket protocol
families. The latter are also known as "domains", as per the first
parameter of the socket(2) API. More specifically, this patch adds
the basic infrastructure for specifying that a particular service is
the socket driver for a set of domains.
Unlike major number mappings for block and character drivers, socket
domain mappings are static. For that reason, they are specified in
system.conf files, using the "domain" keyword. Such a keyword is to
be followed by one or more protocol families, without their "PF_"
prefix. For example, a service with the line "domain INET INET6;"
will be mapped as the socket driver responsible for the AF_INET and
AF_INET6 protocol families.
This patch implements only the infrastructure for creating such
mappings; the actual mapping will be implemented in VFS in a later
patch. The infrastructure is implemented in service(8), RS, and VFS.
For now there is a hardcoded limit of eight domains per socket driver.
This may sound like a lot, but the upcoming new LWIP service will
already use four of those. Also, it is allowed for a service to be
both a block/character driver and a socket driver at the same time,
which is a requirement for the new LWIP service.
Change-Id: I93352d488fc6c481e7079248082895d388c39f2d
show more ...
|
| 77e79d33 | 17-Feb-2017 |
David van Moolenbroek <david@minix3.org> |
etc: synchronize master.password, group to NetBSD
IMPORTANT: this change has a docs/UPDATING entry!
This change is a long overdue switch-over from the old MINIX set of
user and group accounts to th
etc: synchronize master.password, group to NetBSD
IMPORTANT: this change has a docs/UPDATING entry!
This change is a long overdue switch-over from the old MINIX set of
user and group accounts to the NetBSD set. This switch-over is
increasingly important now that we are importing more and more
utilities from NetBSD, several of which expect various user accounts
to exist. By switching over in one go, we save ourselves various
headaches in the long run, even if the switch-over itself is a bit
painful for existing MINIX users.
The newly imported master.passwd and group files have three exceptions
compared to their NetBSD originals:
1. There is a custom "service" account for MINIX 3 services. This
account is used to limit run-time privileges of various system
services, and is not used for any files on disk. Its user ID may
be changed later, but should always correspond to whatever the
SERVICE_UID definition is set to.
2. The user "bin" has its shell set to /bin/sh, instead of NetBSD's
/sbin/nologin. The reason for this is that the test set in
/usr/tests/minix-posix will not be able to run otherwise.
3. The group "operator" has been set to group ID 0, to match its old
value. This tweak is purely for transitioning purposes: as of
writing, pkgsrc packages are still using root:operator as owner and
group for most installed files. Sometime later, we can change back
"operator" to group ID 5 without breaking anything, because it does
not appear that this group name is used for anything important.
Change-Id: I689bcfff4cf7ba85c27d1ae579057fa3f8019c68
show more ...
|