Add integrated dports build feature to build-all and install-all targets

When WITH_DPORTS=YES is used build-all and install-all, these targets will
also build the base set of ports and all of their

Add integrated dports build feature to build-all and install-all targets

When WITH_DPORTS=YES is used build-all and install-all, these targets will
also build the base set of ports and all of their dependencies using
configuration data and the "dragonfly/base" dport found in
"/usr/src/dports.base".

This feature is intended to be used by the nrelease build in an upcoming
commit, and may also be used manually.

The /usr/dports tree must exist and contains the dports repo to use as the
basis of the dports build. It will be null-mounted in the chroot and the
"/usr/src/dports.base/dragonfly" directory will be null-mounted under it
to provide the "dragonfly/base" meta-port. The DPORTSBASE make variable
may be used to override the location of the main dports repo.

The build-all and install-all targets use a sophisticated chrooted
environment to do a completely clean dports build with dsynth. It
It will actually "make installworld" plus a clean "make distribution"
inside the chroot environment to create the environment, and the host
system's /etc/resolv.conf will be copied in.

build-all - Augmented when used WITH_DPORTS=YES
install-all - Augmented when used WITH_DPORTS=YES
buildportschroot - Build a fresh chroot environment for dsynth
mountports - Do all null mounts and cp's needed for the env
buildports - Run dsynth build dragonfly/base in the env
installports - Install the built ports on the host system (see note)
umountports - Remove the null mounts

WITH_DPORTS=YES - executes buildportschroot, mountports, buildports,
and umountports when run from build-all.

executes mountports, installports, umountports
when run from install-all.

DPORTSBASE=path - Defaults to /usr/dports, may be used to override
the location where the main dports tree can be
found.

NOTE: Installports may require interactivity as the base system might already
have installed ports. Generally speaking it is best to not revert any
ports you might already have installed, so feel free to tell it not to
install anything.

* Added various make targets and augmented build-all and install-all

* /usr/local/sbin added and the stale /usr/pkg/bin removed from _HOSTPATH.

* Add /usr/src/dports.base infrastructure.

It contains the dsynth.ini and pkg.conf files that will be used for the
local dsynth operations. Some paths are replaced with SED when copied
to the chroot. The dsynth.ini is relative to the chroot.

It also contains the dragonfly/base dport which is the meta-port that
dsynth will be told to build.

All other dports related files and directories are null-mounted from
/usr/dports.

* Add /usr/distfiles.base to mtree/BSD.usr.dist. This directory is also
created by the build if it does not exist since the host system might
not have run a new installworld recently.

This directory will be used during build-all (buildports) to store the
distfiles required to build dragonfly/base, so they only need to be
fetched once.

* Add the "-C ConfigBase" option to dsynth to make running it from the
chrooted environment easier. This overrides the /etc/dsynth directory
and removes /etc/dsynth as a fallback.

* dsynth conditionalizes calling syscap_set() for backwards compatibility
with older host systems that might not have been upgraded to include the
feature yet.

show more ...

Makefile.inc1: Fix a typo.

boot: Move boot source code /usr/src/stand.

sys/boot becomes stand/boot and lib/libstand becomes stand/lib.

My main reason is to have boot out of the way when for example grepping
in sys/. It also

boot: Move boot source code /usr/src/stand.

sys/boot becomes stand/boot and lib/libstand becomes stand/lib.

My main reason is to have boot out of the way when for example grepping
in sys/. It also makes more sense since it's not really userland or
kernel code.

Other BSDs have moved their boot/standalone code around too, but maybe
with different directory layouts.

Approved-by: dillon, zrj

show more ...

Makefile.inc1: Remove unneeded exists() checks.

Just doing 'make' in /usr/src isn't used for building anymore for a long
time, and our targets don't really make sense with an incomplete sounrce
tree.

Makefile.inc1: Put all backup related stuff under the NO_BACKUP check.

Main effect is that /usr/obj/world_backup is no longer created when
NO_BACKUP is set.

Makefile: Make 'installworld' depend on 'preupgrade'

The 'preupgrade' target will creates new users/groups, which may be
required by the 'installworld' target, so make 'installworld' depend on
'preu

Makefile: Make 'installworld' depend on 'preupgrade'

The 'preupgrade' target will creates new users/groups, which may be
required by the 'installworld' target, so make 'installworld' depend on
'preupgrade'. This will fix the installation of nvmmctl(8) that
requires the 'nvmm' group.

As another result, no longer need to check for users/groups in the
'installcheck' target.

show more ...

build - Add zstd to the build (libzstd as private library)

* Bring in contrib/zstd from FreeBSD.

* Add libzstd as a private library (i.e. priv/libprivate_zstd) to
avoid confusion. (Note that Fre

build - Add zstd to the build (libzstd as private library)

* Bring in contrib/zstd from FreeBSD.

* Add libzstd as a private library (i.e. priv/libprivate_zstd) to
avoid confusion. (Note that FreeBSD also has zstd in base as
a private library).

* Add zstd, unzstd, zstdcat, zstdmt utilities.

* Integrate into libarchive, initrd, dsynth, and misc other
base system elements.

Ported-from: FreeBSD

show more ...

build - Adjust message output after a make build-all

* Clarify that the recommended pkg upgrade -f should be run after
rebooting into the new system.

Switch to ld.bfd as default ld for world stage.

Until boot/efi issues are resolved:
* ld.gold not respecting linker script for section merges;
* clang built boot1.efi and loader.efi failures.

D

Switch to ld.bfd as default ld for world stage.

Until boot/efi issues are resolved:
* ld.gold not respecting linker script for section merges;
* clang built boot1.efi and loader.efi failures.

Default can be restored by setting WORLD_LDVER=ld.gold in make.conf(5).
Final installed ld(1) will still default to ld.gold to not disturb
things in DPorts.

NOTE: this might produce different final binaries if building in tree,
but it should not be fatal as we support both ld.bfd and ld.gold.
Use "make LDVER=ld.bfd" to mimic buildworld behaviour when needed.

show more ...

Switch to binutils234 as default base binutils.

zoneinfo: Update /etc/localtime when upgrading timezones.

* Add tzsetup(8) to the bootstrap-tools.

* Install the backward file to retain obsolete timezones as links
for backward compatibility. Re

zoneinfo: Update /etc/localtime when upgrading timezones.

* Add tzsetup(8) to the bootstrap-tools.

* Install the backward file to retain obsolete timezones as links
for backward compatibility. Remove all these timezones from
Makefile_upgrade.inc again. Also extend mtree/BSD.usr.dist for
this and remove the old 'dragonfly' file (which was only for UTC).

* After installing new timezones, run tzsetup -r to upgrade the user's
chosen timezone. Allow this to fail for quickworlders. It will start
working after the next full buildworld.

Taken-from: FreeBSD (with modifications)

show more ...

Build libssh with ldns support. 2/2

Hook ldns support into libssh via our libprivate_ldns:

- revise libraries build order
- adapt Makefiles
- update libssh/config.h
- drop libssh/openbsd-compat/get

Build libssh with ldns support. 2/2

Hook ldns support into libssh via our libprivate_ldns:

- revise libraries build order
- adapt Makefiles
- update libssh/config.h
- drop libssh/openbsd-compat/getrrsetbyname.c, import
getrrsetbyname-ldns.c instead

Approved by: Rimvydas Jasinskas

show more ...

CURDIR returns an absolute path, so GENLOGDIR does not need this slash in it.

Remove patch(1) from the list of bootstrap tools.

Now that we no longer ship patches in the tree, patch(1) is no longer
needed during buildworld, buildkernel etc. The last patch was removed
in d8d77

Remove patch(1) from the list of bootstrap tools.

Now that we no longer ship patches in the tree, patch(1) is no longer
needed during buildworld, buildkernel etc. The last patch was removed
in d8d77baac2714febf36f4ac37afccbbbdcdc8857.

While here, also remove our old bsd.patch.mk which was used in the
old days to handle local changes to contrib/ code using patches we
shipped. Since we have git, we use vendor branches which we merge and
then just modify on master for this purpose.

show more ...

Makefile.inc1: Pass _SHLIBDIRPREFIX for btools too.

Allow to to link in with crossworld target built libraries if available.

These are not needed for DragonFly crossworld target, however helps wi

Makefile.inc1: Pass _SHLIBDIRPREFIX for btools too.

Allow to to link in with crossworld target built libraries if available.

These are not needed for DragonFly crossworld target, however helps with
bootstrapping on OpenBSD or glibc based systems where static versions
are not available or mismatches the lex(1) too much.

show more ...

Makefile.inc1: Add a missing '@' before an 'echo'.

world: Introduce NO_INITRD option.

Sometimes it is useful to skip rebuilding initrd during development.

Makefile.inc1: Remove id(1) from btools.

No longer required for buildworld/installworld.

Makefile.inc1: Disconnect libelf from the build.

Internal lib is no longer needed after libexecinfo changes.

Remove the old libc_r.

Nothing is using it, if it even works, and no one has been working
on it for a long time.

Approved-by: dillon

build - Add build-all and install-all targets

* Add a 'smart' build-all and install-all target that spews to a
log file instead of to the screen and. Will automatically
run the build stage -j

build - Add build-all and install-all targets

* Add a 'smart' build-all and install-all target that spews to a
log file instead of to the screen and. Will automatically
run the build stage -j N, and both targets can be specified on
one command line.

make build-all install-all

build-all: runs buildworld and buildkernel -j hw.ncpu
install-all: runs installkernel and installworld -j 1

* The usual warnings and cautions still apply when installing a
new world and kernel.

show more ...

nrelease: Simplify by not handling multiple kernel configs

We don't ship multiple kernels for years, so we can get rid of handling
multiple kernel configs in nrelease/Makefile. This simplifies the

nrelease: Simplify by not handling multiple kernel configs

We don't ship multiple kernels for years, so we can get rid of handling
multiple kernel configs in nrelease/Makefile. This simplifies the
Makefile and also get rid of the hack of 'SUBDIR += nrelease' in
Makefile.inc1.

Note that users can still override 'KERNCONF' the same as 'buildkernel'.

show more ...

Generate a fresh /etc/os-release on each installworld.

Reported-by: zrj

Nuke libradius(3).

It was moved to DPorts.

For now keep radlib.c code directly in ppp(8). Other BSDs have it atm.

Nuke libtacplus(3).

It was moved to DPorts.