buildworld: Add support to skip toolchain if requested.

These two new features are mainly aimed at developers, to speed up their
rebuilds on development branches. Often there are no changes that wou

buildworld: Add support to skip toolchain if requested.

These two new features are mainly aimed at developers, to speed up their
rebuilds on development branches. Often there are no changes that would
influence the toolchain components in any way. So it is reasonable to
skip rebuild of certain parts (mostly to avoid building c++ sources).

Previously we were rebuilding both main and alternative compilers twice
during buildworld for a full bootstrap that allowed us to avoid certain
breakages and not to keep adding lots of backwards compatibility hacks.

The new additions:
* -DNO_CROSSTOOLS - skip compiler and binutils bootstrap;
* -DNO_TOOLCHAINS - do not build compilers/binutils for installworld.

Both features are aimed for testing DESTDIR targets. Decision to
activate any of these features is left to the user's own discretion
based on reviewing the commits to the git repository or local clone.

The NO_CROSSTOOLS feature is implemented by installing the shell script
wrappers that add a sysroot and patch the specs. Idea is not to disturb
in any way the Makefile.inc1 or objformat logic at this moment. Since
all the main toolchain tools are installed as shell scripts this has a
nice bonus for quick hooks/probes and other hacky/pretty stuff without
disturbing the rest of the buildworld. You can even add your favorite
static analyser tool, just don't forget to change default include path
to /usr/obj/../world_x86_64/usr/include for it.

The objformat changes for sysroot would only make sense when we make a
port for another architecture. None of the new features are considered
to be guessTM enabled automatically at this moment. To be sure that
nothing has changed since previous buildworld from other changes in the
source tree one would have to compile and hex-diff the final products.
Keep in mind that libraries coming with the compilers will be rebuilt
even with both knobs passed. This should help catching most of the
issues before installworld step and not on the next {build,quick}world.

Both knobs are not documented in make.conf(5) on purpose. It is highly
advised *not* adding any of these knobs into make.conf, sooner or later
you will forget it and *will* get yourself in world recovery situation:
a) world backup machinery (unless you had NO_BACKUP=true in make.conf);
b) perform manual bootstrap to fix up the breakage;
c) restore from H1/H2 snapshots (toolchain components are located in
/usr/libexec/* and /usr/lib/${CCVER}/ directories);
d) clean overwrite by mounting from other system (like usb img stick).

Features should work with quickworld too.
To put things in perspective here is a little chart of the cleaned
fresh 'make -jX buildworld' times impact on few setups:

netbook: hw.model: Intel(R) Core(TM) i3 CPU U380 @ 1.33GHz
compiler: gcc50 as cc(1) (4gb usb stick for /, ironlake cpu)
make -j3 buildworld [-DNO_CROSSTOOLS] [-DNO_TOOLCHAIN]

min:sec | STANDARD | NO_C | NO_T | NO_C/NO_T |
----------------------------------------------------------------
gcc50 | 73:56 | 59:57 | 54:11 | 39:38 |
gcc47 (host+world)| N/I | N/I | N/I | 37:51 |
----------------------------------------------------------------

laptop setup: hw.model: Intel(R) Core(TM) i7-4712MQ CPU @ 2.30GHz
compiler: gcc50 as cc(1) (default)
make -j9 buildworld [-DNO_CROSSTOOLS] [-DNO_TOOLCHAIN]

min:sec | STANDARD | NO_C | NO_T | NO_C/NO_T |
----------------------------------------------------------------
gcc50 | 17:06 | 13:57 | 12:58 | 09:42 |
gcc50 gcc47 (def) | 20:18 | 16:03 | 14:58 | 10:37 |
gcc50 gcc80 | 24:24 | 18:06 | 17:31 | 11:23 |
gcc50 gcc47 gcc80 | 27:47 | 20:17 | 19:23 | 12:00 |
----------------------------------------------------------------

blade setup: dual hw.model: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
compiler: gcc80 as cc(1) (soon gcc80 will replace gcc50 in master)
make -j32 buildworld [-DNO_CROSSTOOLS] [-DNO_TOOLCHAIN]

min:sec | STANDARD | NO_C | NO_T | NO_C/NO_T |
----------------------------------------------------------------
gcc80 | 06:58 | 05:19 | 05:33 | 03:51 |
gcc80 gcc47 (def*)| 08:03 | 06:06 | 06:02 | 04:05 |
gcc80 gcc50 | 08:23 | 06:12 | 06:19 | 04:08 |
gcc80 gcc47 gcc50 | 09:24 | 06:59 | 06:51 | 04:21 |
----------------------------------------------------------------

show more ...