Syn cache calls TCP drop instead of socket abort.

Instead of calling socket layer soabort() and then down via tcp_abort()
which ends in tcp_drop(), call tcp_drop() directly from the TCP syn
cache.

Syn cache calls TCP drop instead of socket abort.

Instead of calling socket layer soabort() and then down via tcp_abort()
which ends in tcp_drop(), call tcp_drop() directly from the TCP syn
cache. The errno is not relevant as the new socket is dropped
before it can be reached from userland.

OK mvs@

show more ...

Rename old socket to more specific listen socket in TCP syn cache.

OK mvs@

Keep socket lock in sonewconn() for new connection.

For TCP input unlocking we need a consistent lock of the newly
created socket. Instead of releasing the lock in sonewconn() and
grabbing it again

Keep socket lock in sonewconn() for new connection.

For TCP input unlocking we need a consistent lock of the newly
created socket. Instead of releasing the lock in sonewconn() and
grabbing it again later, it is better that sonewconn() returns a
locked socket.
For now only change syn_cache_get() which calls in_pcbsounlock_rele()
at the end. Following diffs will push the unlock into tcp_input().

OK mvs@

show more ...

Remove net lock from TCP sysctl for keep alive.

Keep copies in seconds for the sysctl and update timer variables
atomically when they change. tcp_maxidle was historically calculated
in tcp_slowtimo

Remove net lock from TCP sysctl for keep alive.

Keep copies in seconds for the sysctl and update timer variables
atomically when they change. tcp_maxidle was historically calculated
in tcp_slowtimo() as the timers were called from there. Better
calculate maxidle when needed. tcp_timer_init() is useless, just
initialize data. While there make the names consistent.

input sthen@; OK mvs@

show more ...

Fix indent.

Run TCP sysctl ident and drop with shared net lock.

Convert exclusive net lock for TCPCTL_IDENT and TCPCTL_DROP to
shared net lock and push it down into tcp_ident(). Grab the socket
lock there with

Run TCP sysctl ident and drop with shared net lock.

Convert exclusive net lock for TCPCTL_IDENT and TCPCTL_DROP to
shared net lock and push it down into tcp_ident(). Grab the socket
lock there with in_pcbsolock_ref(). Move socket release from
in_pcbsolock() to in_pcbsounlock_rele() and add _ref and _rele
suffix to the inpcb socket lock functions. They both lock and
refcount now. in_pcbsounlock_rele() ignores NULL sockets to make
the unlock path in error case simpler. Socket lock also protects
tcp_drop() and tcp_close() now, so the socket pointer from incpb
may be NULL during unlock. In tcp_ident() improve consistency check
of address family.

OK mvs@

show more ...

Read tcp_do_ecn only once during packet processing and mark atomic.

OK mvs@

Use atomic operations to modify the MTU of route.

When unlocking TCP, path MTU discovery will run in parallel. To
keep route MTU consistent, make access to rt_mtu atomic. Use
compare-and-swap func

Use atomic operations to modify the MTU of route.

When unlocking TCP, path MTU discovery will run in parallel. To
keep route MTU consistent, make access to rt_mtu atomic. Use
compare-and-swap function to detect whether another thread is
modifying the MTU field. In this case skip updating rt_mtu.

OK mvs@

show more ...

Run TCP SYN cache timer with shared net lock.

Reset the inp_socket pointer in in_pcbdetach() when a socket is
about to be freed. By protecting this with a mutex, it is possible
to reliable retrieve

Run TCP SYN cache timer with shared net lock.

Reset the inp_socket pointer in in_pcbdetach() when a socket is
about to be freed. By protecting this with a mutex, it is possible
to reliable retrieve the socket from an inpcb reference. By using
the socket refcount of the listen socket, switch from mutex to
socket lock in syn_cache_timer().

tested by Hrvoje Popovski; OK mvs@

show more ...

Reference count the listen inpcb in the TCP SYN cache.

To make progress in unlocking TCP input path, more reference counting
is needed. The SYN cache has a reference to the listen socket in
form of

Reference count the listen inpcb in the TCP SYN cache.

To make progress in unlocking TCP input path, more reference counting
is needed. The SYN cache has a reference to the listen socket in
form of a struct tcpcb. Instead of adding a refcount to tcpcb, it
is easier to use a struct inpcb pointer which already has a refcount.
Acquire and hold the reference while running SYN cache timer.

OK mvs@

show more ...

Use per-sockbuf mutex(9) to protect `so_snd' buffer of tcp(4) sockets.

Even for tcp(4) case, sosend() only checks `so_snd' free space and
sleeps if necessary, actual buffer handling happens in soloc

Use per-sockbuf mutex(9) to protect `so_snd' buffer of tcp(4) sockets.

Even for tcp(4) case, sosend() only checks `so_snd' free space and
sleeps if necessary, actual buffer handling happens in solock()ed PCB
layer.

Only unlock sosend() path, the somove() is still locked exclusively. The
"if (dosolock)" dances are useless, but intentionally left as is.

Tested and ok by bluhm.

show more ...

Remove net lock from TCP syn cache sysctl.

TCP syn cache is protected by mutex. Make access to its sysctl
variables either atomic or put them into this mutex. Then net lock
can be removed.

OK mvs@

Read more TCP sysctl variables atomically.

OK mvs@

Declare TCP variable tcprexmtthresh const.

OK mvs@

Protect send buffer with mutex in tcp_mss_update().

To prepare for running tcp_output() in parallel, protect socket
send buffer high water field and sbreserve() with mutex.

OK mvs@

Make tcp_mss() MP safe so that it could be called with socket lock.

Read tcp_mssdflt and tcp_do_rfc3390 atomically. Address family
must be AF_INET or AF_INET6, panic if not. This makes goto out a

Make tcp_mss() MP safe so that it could be called with socket lock.

Read tcp_mssdflt and tcp_do_rfc3390 atomically. Address family
must be AF_INET or AF_INET6, panic if not. This makes goto out a
bit simpler. Route MTU must be read once, another thread might
modify it. Fix the signed versus unsigned comparison with max()
and min().

OK mvs@

show more ...

Fix signed integer comparison in tcp mss.

In tcp_mss_adv() max(9) was used to guarantee that mss it not too
small. Unfortunately max() uses u_int and mss could get negative
in some error conditions

Fix signed integer comparison in tcp mss.

In tcp_mss_adv() max(9) was used to guarantee that mss it not too
small. Unfortunately max() uses u_int and mss could get negative
in some error conditions.
Rearrange the code to directly return in case of errors. Also read
tcp_mssdflt only once to head towards atomic integer sysctl.

OK mvs@

show more ...

Use per-sockbuf mutex(9) to protect `so_rcv' buffer of tcp(4) sockets.

Only unlock soreceive() path, somove() path still locked exclusively. Also
exclusive socket lock will be taken in the soreceiv

Use per-sockbuf mutex(9) to protect `so_rcv' buffer of tcp(4) sockets.

Only unlock soreceive() path, somove() path still locked exclusively. Also
exclusive socket lock will be taken in the soreceive() path each time
before pru_rcvd() call.

Note, both socket and `sb_mtx' locks are held while SS_CANTRCVMORE
modified, so socket lock is enough to check it in the protocol input
path.

ok bluhm

show more ...

Use mutex of receive socket buffer to protect so_oobmark.

Socket field so_oobmark belongs to receive path, so use so_rcv mutex
to protect it. Although tcp_input() is still exclusively locked,
put m

Use mutex of receive socket buffer to protect so_oobmark.

Socket field so_oobmark belongs to receive path, so use so_rcv mutex
to protect it. Although tcp_input() is still exclusively locked,
put mutex there to prepare further unlocking.

OK mvs@

show more ...

Rearrange #ifdef TCP_SIGNATURE to keep braces balanced.

remove unused packet header length defines

Use struct ipsec_level within inpcb.

Instead of passing around u_char[4], introduce struct ipsec_level
that contains 4 ipsec levels. This provides better type safety.
The embedding struct inpcb is

Use struct ipsec_level within inpcb.

Instead of passing around u_char[4], introduce struct ipsec_level
that contains 4 ipsec levels. This provides better type safety.
The embedding struct inpcb is globally visible for netstat(1), so
put struct ipsec_level outside of #ifdef _KERNEL.

OK deraadt@ mvs@

show more ...

correct indentation

no functional change, found by smatch warnings
ok miod@ bluhm@

Split single TCP inpcb table into IPv4 and IPv6 parts.

With two separate TCP hash tables, each one becomes smaller. When
we remove the exclusive net lock from TCP, contention on internet
PCB table

Split single TCP inpcb table into IPv4 and IPv6 parts.

With two separate TCP hash tables, each one becomes smaller. When
we remove the exclusive net lock from TCP, contention on internet
PCB table mutex will be reduced. UDP has been split earlier into
IPv4 and IPv6. Replace branch conditions based on INP_IPV6 with
assertions.

OK mvs@

show more ...

Move global variables for TCP debug onto the tcp_input() stack.

OK mvs@