[Multilib] Custom flags processing for library selection (#110659)

This patch is the third step to extend the current multilib system to
support the selection of library variants which do not corre

[Multilib] Custom flags processing for library selection (#110659)

This patch is the third step to extend the current multilib system to
support the selection of library variants which do not correspond to
existing command-line options.

Proposal can be found in
https://discourse.llvm.org/t/rfc-multilib-custom-flags/81058

The multilib mechanism supports libraries that target code generation or
language options such as --target, -mcpu, -mfpu, -mbranch-protection.
However, some library variants are particular to features that do not
correspond to any command-line options. Examples include variants for
multithreading and semihosting.

This work introduces a way to instruct the multilib system to consider
these features in library selection. This particular patch is comprised
of the core processing of these flags.

- Custom flags in the command-line are read and forwarded to the
multilib system. If multiple flag values are present for the same flag
declaration, the last one wins. Default flag values are inserted for
flag declarations for which no value was given.
- Feed `MacroDefines` back into the driver. Each item `<string>` in the
`MacroDefines` list is formatted as `-D<string>`.

Library variants should list their requirement on one or more custom
flags like they do for any other flag. The new command-line option is
passed as-is to the multilib system, therefore it should be listed in
the format `-fmultilib-flag=<str>`.

Moreover, a variant that does not specify a requirement on any
particular flag can be matched against any value of that flag.

If the user specifies `-fmultilib-flag=<name>` with a name that is
invalid, but close enough to any valid flag value name in terms of edit
distance, a suggesting error is shown:
```
error: unsupported option '-fmultilib-flag=invalidname'; did you mean '-fmultilib-flag=validname'?
```
The candidate with the smallest edit distance is chosen for the
suggestion, up to a certain maximum value (implementation detail), after
which a non-suggesting error is shown instead:
```
error: unsupported option '-fmultilib-flag=invalidname'
```

show more ...

[Multilib] Custom flags YAML parsing (#122903)

This patch is the first step to extend the current multilib system to
support the selection of library variants which do not correspond to
existing c

[Multilib] Custom flags YAML parsing (#122903)

This patch is the first step to extend the current multilib system to
support the selection of library variants which do not correspond to
existing command-line options.

Proposal can be found in
https://discourse.llvm.org/t/rfc-multilib-custom-flags/81058

The multilib mechanism supports libraries that target code generation or
language options such as `--target`, `-mcpu`, `-mfpu`,
`-mbranch-protection`. However, some library variants are particular to
features that do not correspond to any command-line options. Examples
include variants for multithreading and semihosting.

This work introduces a way to instruct the multilib system to consider
these features in library selection. This particular patch comprises a
new section in `multilib.yaml` to declare flags for which no option
exists. Henceforth this sort of flag will be called `custom flag` for
clarity.

The `multilib.yaml` file will have a new section called Flags which
contains the declarations of the target’s custom flags:

```yaml
Flags:
- Name: multithreaded
Values:
- Name: no-multithreaded
MacroDefines: [__SINGLE_THREAD__]
- Name: multithreaded
Default: no-multithreaded

- Name: io
Values:
- Name: io-none
- Name: io-semihosting
MacroDefines: [SEMIHOSTING]
- Name: io-linux-syscalls
MacroDefines: [LINUX_SYSCALLS, HOSTED=1]
Default: io-none
```
- Name: the name to categorize a flag.
- Values: a list of possible values.
- Default: it specifies which value this flag should take if not
specified in the command-line invocation. It must be one value from the
Values field.

Each flag Value follows this description:
- Name (required): the name of the custom flag value (string). This is
the string to be used in `-fmultilib-flag=<string>`.
- MacroDefines (optional): a list of strings to be used as macro
definitions. Each string
is fed into the driver as ``-D<string>``.

A Default value is useful to save users from specifying custom flags
that have a most commonly used value.

The namespace of flag values is common across all flags. This means that
flag values must be unique.

show more ...

Revert "[Multilib] Custom flags YAML parsing" (#122722)

Reverts llvm/llvm-project#110657

It seems that this patch is causing the sanitizer bot to fail. Reverting
while I investigate

[Multilib] Custom flags YAML parsing (#110657)

This patch is the first step to extend the current multilib system to
support the selection of library variants which do not correspond to
existing c

[Multilib] Custom flags YAML parsing (#110657)

This patch is the first step to extend the current multilib system to
support the selection of library variants which do not correspond to
existing command-line options.

Proposal can be found in
https://discourse.llvm.org/t/rfc-multilib-custom-flags/81058

The multilib mechanism supports libraries that target code generation or
language options such as `--target`, `-mcpu`, `-mfpu`,
`-mbranch-protection`. However, some library variants are particular to
features that do not correspond to any command-line options. Examples
include variants for multithreading and semihosting.

This work introduces a way to instruct the multilib system to consider
these features in library selection. This particular patch comprises a
new section in `multilib.yaml` to declare flags for which no option
exists. Henceforth this sort of flag will be called `custom flag` for
clarity.

The `multilib.yaml` file will have a new section called Flags which
contains the declarations of the target’s custom flags:

```yaml
Flags:
- Name: multithreaded
Values:
- Name: no-multithreaded
MacroDefines: [__SINGLE_THREAD__]
- Name: multithreaded
Default: no-multithreaded

- Name: io
Values:
- Name: io-none
- Name: io-semihosting
MacroDefines: [SEMIHOSTING]
- Name: io-linux-syscalls
MacroDefines: [LINUX_SYSCALLS, HOSTED=1]
Default: io-none
```
- Name: the name to categorize a flag.
- Values: a list of possible values.
- Default: it specifies which value this flag should take if not
specified in the command-line invocation. It must be one value from the
Values field.

Each flag Value follows this description:
- Name (required): the name of the custom flag value (string). This is
the string to be used in `-fmultilib-flag=<string>`.
- MacroDefines (optional): a list of strings to be used as macro
definitions. Each string
is fed into the driver as ``-D<string>``.

A Default value is useful to save users from specifying custom flags
that have a most commonly used value.

The namespace of flag values is common across all flags. This means that
flag values must be unique.

show more ...

[Driver] Remove unused includes (NFC) (#116316)

Identified with misc-include-cleaner.

[clang][Driver] Improve multilib custom error reporting (#110804)

If `multilib.yaml` reports a custom error message for some unsupported
configuration, it's not very helpful to display that error me

[clang][Driver] Improve multilib custom error reporting (#110804)

If `multilib.yaml` reports a custom error message for some unsupported
configuration, it's not very helpful to display that error message
_first_, and then follow it up with a huge list of all the multilib
configurations that _are_ supported.

In interactive use, the list is likely to scroll the most important
message off the top of the user's window, leaving them with just a
long list of available libraries, without a visible explanation of
_why_ clang just printed that long list. Also, in general, it makes
more intuitive sense to print the message last that shows why
compilation can't continue, because that's where users are most likely
to look for the reason why something stopped.

show more ...

[clang][Driver] Rename "FatalError" key to "Error" in multilib.yaml (#110804)

This is a late-breaking change to #105684, suggested after the
original patch was already landed.

[clang][Driver] Add a custom error option in multilib.yaml. (#105684)

Sometimes a collection of multilibs has a gap in it, where a set of
driver command-line options can't work with any of the avai

[clang][Driver] Add a custom error option in multilib.yaml. (#105684)

Sometimes a collection of multilibs has a gap in it, where a set of
driver command-line options can't work with any of the available
libraries.

For example, the Arm MVE extension requires special startup code (you
need to initialize FPSCR.LTPSIZE), and also benefits greatly from
-mfloat-abi=hard. So a multilib provider might build a library for
systems without MVE, and another for MVE with -mfloat-abi=hard,
anticipating that that's what most MVE users would want. But then if a
user compiles for MVE _without_ -mfloat-abi=hard, thhey can't use either
of those libraries – one has an ABI mismatch, and the other will fail to
set up LTPSIZE.

In that situation, it's useful to include a multilib.yaml entry for the
unworkable intermediate situation, and have it map to a fatal error
message rather than a set of actual libraries. Then the user gets a
build failure with a sensible explanation, instead of selecting an
unworkable library and silently generating bad output. The new
regression test demonstrates this case.

This patch introduces extra syntax into multilib.yaml, so that a record
in the `Variants` list can omit the `Dir` key, and in its place, provide
a `FatalError` key. Then, if that variant is selected, the error message
is emitted as a clang diagnostic, and multilib selection fails.

In order to emit the error message in `MultilibSet::select`, I had to
pass a `Driver &` to that function, which involved plumbing one through
to every call site, and in the unit tests, constructing one specially.

show more ...

[Driver] Use SmallVectorImpl reference instead of SmallVector reference in MultilibSet. NFC

We prefer to pass SmallVectorImpl reference so that no specific
inline size is required.

[Driver] Add exclusive-group feature to multilib.yaml. (#69447)

This allows a YAML-based multilib configuration to specify explicitly
that a subset of its library directories are alternatives to ea

[Driver] Add exclusive-group feature to multilib.yaml. (#69447)

This allows a YAML-based multilib configuration to specify explicitly
that a subset of its library directories are alternatives to each
other, i.e. at most one of that subset should be selected.

So if you have multiple sysroots each including a full set of headers
and libraries, you can mark them as members of the same mutually
exclusive group, and then you'll be sure that only one of them is
selected, even if two or more are compatible with the compile options.

This is particularly important in multilib setups including the libc++
headers, where selecting the include directories from two different
sysroots can cause an actual build failure. This occurs when including
<stdio.h>, for example: libc++'s stdio.h is included first, and will
try to use `#include_next` to fetch the underlying libc's version. But
if there are two include directories from separate multilibs, then
both of their C++ include directories will end up on the include path
first, followed by both the C directories. So the `#include_next` from
the first libc++ stdio.h will include the second libc++ stdio.h, which
will do nothing because it has the same include guard macro, and the
libc header won't ever be included at all.

If more than one of the options in an exclusive group matches the
given flags, the last one wins.

The syntax for specifying this in multilib.yaml is to define a Groups
section in which you specify your group names, and for each one,
declare it to have Type: Exclusive. (This reserves space in the syntax
for maybe adding other group types later, such as a group of mutually
_dependent_ things that you must have all or none of.) Then each
Variant record that's a member of a group has a Group: property giving
that group's name.

show more ...

[Driver] Use llvm::any_of (NFC)

[clang] Stop including llvm/ADT/StringMap.h (NFC)

These source files do not use StringMap.h.

[Driver] Enable selecting multiple multilibs

This will enable layering multilibs on top of each other.
For example a multilib containing only a no-exceptions libc++ could be
layered on top of a mult

[Driver] Enable selecting multiple multilibs

This will enable layering multilibs on top of each other.
For example a multilib containing only a no-exceptions libc++ could be
layered on top of a multilib containing C libs. This avoids the need
to duplicate the C library for every libc++ variant.

This change doesn't expose the functionality externally, it only opens
the functionality up to be potentially used by ToolChain classes.

Differential Revision: https://reviews.llvm.org/D143059

show more ...

[Driver] Multilib YAML parsing

The format includes a ClangMinimumVersion entry to avoid a potential
source of subtle errors if an older version of Clang were to be used
with a multilib.yaml that req

[Driver] Multilib YAML parsing

The format includes a ClangMinimumVersion entry to avoid a potential
source of subtle errors if an older version of Clang were to be used
with a multilib.yaml that requires a newer Clang to work correctly.
This feature is comparable to CMake's cmake_minimum_required.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D142932

show more ...

[NFC][Driver] Change Multilib flag representation

This new representation means that a valid command line option may
potentially be used directly as a multilib flag without any translation.

To indi

[NFC][Driver] Change Multilib flag representation

This new representation means that a valid command line option may
potentially be used directly as a multilib flag without any translation.

To indicate that a flag is required not to be present, its first
character is replaced with '!', which is intended for consistency with
the logical not operator in many programming languages.

Reviewed By: simon_tatham

Differential Revision: https://reviews.llvm.org/D151438

show more ...

[Driver] Change multilib selection algorithm

The new algorithm is:
1. Find all multilibs with flags that are a subset of the requested
flags.
2. If more than one multilib matches, choose the last

[Driver] Change multilib selection algorithm

The new algorithm is:
1. Find all multilibs with flags that are a subset of the requested
flags.
2. If more than one multilib matches, choose the last.

In addition a new selection mechanism is permitted via an overload of
MultilibSet::select() for which multiple multilibs are returned.
This allows layering multilibs on top of each other.

Since multilibs are now ordered within a list, they no longer need a
Priority field.

The new algorithm is different to the old algorithm, but in practise
the old algorithm was always used in such a way that the effect is the
same.
The old algorithm was to find the set intersection of the requested
flags (with the first character of each removed) with each multilib's
flags (ditto), and for that intersection check whether the first
character matched. However, ignoring the first characters, the
requested flags were always a superset of all the multilibs flags.
Therefore the new algorithm can be used as a drop-in replacement.

The exception is Fuchsia, which needs adjusting slightly to set both
fexceptions and fno-exceptions flags.

Differential Revision: https://reviews.llvm.org/D142905

show more ...

[clang] Use *{Set,Map}::contains (NFC)

[NFC] Class for building MultilibSet

The functionality in MultilibSet for creating it is tied to its current
implementation. Putting that code in a separate class is an enabler for
changing the Mult

[NFC] Class for building MultilibSet

The functionality in MultilibSet for creating it is tied to its current
implementation. Putting that code in a separate class is an enabler for
changing the MultilibSet implementation.

Differential Revision: https://reviews.llvm.org/D142893

show more ...

Use the range-based overload of llvm::sort where possible

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D130403

[clang] Use llvm::erase_if (NFC)

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

[Driver] Support priority for multilibs

When more than one multilib flag matches, try to select the best
possible match based on priority. When two different multilibs with
the same same priority ma

[Driver] Support priority for multilibs

When more than one multilib flag matches, try to select the best
possible match based on priority. When two different multilibs with
the same same priority match, we still throw an error matching the
existing behavior.

Differential Revision: https://reviews.llvm.org/D60990

llvm-svn: 359359

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[Driver] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC).

llvm-svn: 328044

Multilib: add dump methods

llvm-svn: 302296