Adjust the reader callback API to return an ibuf and to also claim the
fd which is passed as argument.

This is needed because on Linux the control messages used to pass fds
are not acting as a barri

Adjust the reader callback API to return an ibuf and to also claim the
fd which is passed as argument.

This is needed because on Linux the control messages used to pass fds
are not acting as a barrier and ensuring that the fd is passed with the
first byte of the read call. Instead we need to mark the message that
holds to fd and the scan for that message in the stream.

While there also adjust imsgbuf_set_maxsize() to return an int to
indicate an error if the requested size is out of range.

Problem reported and fix tested by nicm@ on a linux system.
OK tb@

show more ...

Bring the weird msgbuf_free() into a slightly more usual form

ok claudio

Fix NULL deref in msgbuf_free()

From Henry Ford

Add ibuf_read and msgbuf_read to read from a socket and msgbuf_get
to return the ibufs generated by the previous two functions.

Error out if the hdrsz argument in msgbuf_new_reader is 0 or too big.

Add ibuf_read and msgbuf_read to read from a socket and msgbuf_get
to return the ibufs generated by the previous two functions.

Error out if the hdrsz argument in msgbuf_new_reader is 0 or too big.
Also check that the rbuf is allocated in ibuf_read and msgbuf_read.
If not return EINVAL.

Implement the imsg API using these functions and introduce
imsgbuf_set_maxsize() to alter the maximum message size and
imsgbuf_allow_fdpass() to allow fd passing (which is now off by default).
Also cleanup the internals a bit and make imsgbuf_init() return int.

OK tb@

show more ...

Make struct msgbuf opaque. Introduce msgbuf_new() and msgbuf_free() for that.

This does not yet fix the imsgbuf_init() function which can now error.
OK tb@

Remove fd from struct msgbuf, instead pass the fd to imsg_write and
msgbuf_write

OK tb@

Reshuffle code and rename some internal functions.

OK tb@

Retire ibuf_add_buf, replaced by better named ibuf_add_ibuf

OK tb@

Simplify imsg_write, msgbuf_write and ibuf_write return codes.

Return 0 on success or when a temporary error happened (EAGAIN, ENOBUFS).
Return -1 on error and set errno otherwise.
Kill the old 0 re

Simplify imsg_write, msgbuf_write and ibuf_write return codes.

Return 0 on success or when a temporary error happened (EAGAIN, ENOBUFS).
Return -1 on error and set errno otherwise.
Kill the old 0 return for EOF. This is not how write operations work.

OK tb@

show more ...

Allow 0 sized ibuf_open() and mark stack buffers by setting the fd to -2.

For imsgs we want to be able to use ibufs even for empty messages and stash
the fd into those ibufs. For that adjust the ibu

Allow 0 sized ibuf_open() and mark stack buffers by setting the fd to -2.

For imsgs we want to be able to use ibufs even for empty messages and stash
the fd into those ibufs. For that adjust the ibuf code to allow that.
This adds an internal IBUF_FD_MARK_ON_STACK define that is now used
for on stack ibufs instead of setting max to 0.

OK tb@

show more ...

nline ibuf_realloc() in ibuf_reserve() it is the only
user and the function is simple enough.

OK tb@

Add ibuf_get_string() to the ibuf API (used by bgpd and xlockmore)

OK tb@

Replace recallocarray() with a realloc() + memset() combo.

recallocarray(), with its guarantee that memory becoming unallocated is
explicitly discarded, is too slow. In rpki-client forming one parti

Replace recallocarray() with a realloc() + memset() combo.

recallocarray(), with its guarantee that memory becoming unallocated is
explicitly discarded, is too slow. In rpki-client forming one particular
ibuf takes more then 4mins because every recallocarray() call ends up
doing a fresh malloc + memcpy + freezero call.
For sensitive data use ibuf_open() instead of ibuf_dynamic() to avoid
any memory reallocations.
OK tb@

show more ...

Extend imsg and ibuf API with useful getter methods

For ibufs:
- various getters for ibufs (ibuf_get* and ibuf_skip)
- additional ibuf set/add functions that don't alter byte order
- ibuf_truncate a

Extend imsg and ibuf API with useful getter methods

For ibufs:
- various getters for ibufs (ibuf_get* and ibuf_skip)
- additional ibuf set/add functions that don't alter byte order
- ibuf_truncate and ibuf_rewind
- ibuf_from_buffer and ibuf_from_ibuf to populate a reader ibuf
- a getter for the msgbuf queuelen

For imsg:
- various getters for imsg (especially imsg_get_data() which can be used
in most cases as a simple one call api with all error checks).
All the imsg.hdr fields can also be accessed by getters.
- The imsg data is now actually an ibuf but the old imsg.data pointer is
kept for now to not break every imsg application.
- Introduce imsg_forward to simply forward a message from one channel to
an other (used in the control socket code).

Since this requires a major bump take the oportunity to also cleanup some
function signatures to use size_t for length fields. Also internal data
structures are removed from the public header.

With and OK tb@

show more ...

Require callers to use ibuf_fd_get() to extract the passed fd from an ibuf.

In imsg_free() close any fd that was not claimed automatically to prevent
filedescriptor leaks. In ibuf_dequeue() remvoe c

Require callers to use ibuf_fd_get() to extract the passed fd from an ibuf.

In imsg_free() close any fd that was not claimed automatically to prevent
filedescriptor leaks. In ibuf_dequeue() remvoe code which is now part of
imsg_free().
OK tb@

show more ...

Improve the ibuf API by adding these functions:
Functions extending ibuf_add to work with more specific data types
ibuf_add_buf, ibuf_add_zero, ibuf_add_n8, ibuf_add_n16, ibuf_add_n32,
ibuf_add_n

Improve the ibuf API by adding these functions:
Functions extending ibuf_add to work with more specific data types
ibuf_add_buf, ibuf_add_zero, ibuf_add_n8, ibuf_add_n16, ibuf_add_n32,
ibuf_add_n64
Functions replacing ibuf_seek where data at a specific offset is modified
ibuf_set, ibuf_set_n8, ibuf_set_n16, ibuf_set_n32, ibuf_set_n64
Functions to check, get and set the filedescriptor stored on the ibuf
ibuf_fd_avail, ibuf_fd_get, ibuf_fd_set
and ibuf_data() to access the data buffer, to be used together with ibuf_size()

On top of this add an optimized imsg_compose_ibuf() where an ibuf is wrapped
into an imsg in an efficent way.

Finally remove msgbuf_drain since it is not used by anything outside of
the ibuf code. Because of this removal bump the major of libutil.

Remove ibuf_data() in iked since the same function is now provided by libutil.
OK tb@

show more ...

Avoid calling malloc with a zero length argument.

ibuf_open() will return an error in this case while ibuf_dynamic() accepts
a 0 len argument and just initialized the buffer and length to zero.
A la

Avoid calling malloc with a zero length argument.

ibuf_open() will return an error in this case while ibuf_dynamic() accepts
a 0 len argument and just initialized the buffer and length to zero.
A later ibuf_realloc() call will take care of allocating the buffer.

Additionally switch from malloc() to calloc() when allocating the buffer
this way the buffer is initalized and in ibuf_reserve() an addtional
memset() is used to make sure that the reserved data is zeroed.

OK tb@

show more ...

Verify sizes before arithmetic operations

Unsigned overflows are not a bug in C but we have to make sure that
requested buffer sizes will be actually available.

If not, set errno to ERANGE and retu

Verify sizes before arithmetic operations

Unsigned overflows are not a bug in C but we have to make sure that
requested buffer sizes will be actually available.

If not, set errno to ERANGE and return an error value.

ok deraadt, millert

show more ...

change the barrier so that fd's are always passed and received with
the first byte of the imsg they belong to.

idea, tweaks and ok claudio@

Change imsg header definitions to use standard types.

ok deraadt@ claudio@

Make a few internal symbols static and add a Symbols.map version script
to control which symbols are exported from the shared library.

ok guenther@, deraadt@, jca@

Use freezero(3) for the imsg framework in imsg_free(3) and ibuf_free(3).

In our privsep model, imsg is often used to transport sensitive
information between processes. But a process might free an i

Use freezero(3) for the imsg framework in imsg_free(3) and ibuf_free(3).

In our privsep model, imsg is often used to transport sensitive
information between processes. But a process might free an imsg, and
reuse the memory for a different thing. iked uses some
explicit_bzero() to clean imsg-buffer but doing it in the library with
the freezero() is less error-prone and also benefits other daemons.

OK deraadt@ jsing@ claudio@

show more ...

Grow buffers using recallocarray, to avoid the potential dribble that
the standard realloc*() functions can leave behind. imsg buffers are
sometimes used in protocol stacks which require some secrec

Grow buffers using recallocarray, to avoid the potential dribble that
the standard realloc*() functions can leave behind. imsg buffers are
sometimes used in protocol stacks which require some secrecy, and layering
violations would be needed to resolve this issue otherwise.
Discussed with many.

show more ...

check for NULL in ibuf_free().
ok and slight improvement, mmcco@
ok semarie@ and encouragement tedu@ krw@

Use memset instead of bzero for better portability.

ok gilles claudio doug