Rework the `Option` library to reduce dynamic relocations (#119198)

Apologies for the large change, I looked for ways to break this up and
all of the ones I saw added real complexity. This change f

Rework the `Option` library to reduce dynamic relocations (#119198)

Apologies for the large change, I looked for ways to break this up and
all of the ones I saw added real complexity. This change focuses on the
option's prefixed names and the array of prefixes. These are present in
every option and the dominant source of dynamic relocations for PIE or
PIC users of LLVM and Clang tooling. In some cases, 100s or 1000s of
them for the Clang driver which has a huge number of options.

This PR addresses this by building a string table and a prefixes table
that can be referenced with indices rather than pointers that require
dynamic relocations. This removes almost 7k dynmaic relocations from the
`clang` binary, roughly 8% of the remaining dynmaic relocations outside
of vtables. For busy-boxing use cases where many different option tables
are linked into the same binary, the savings add up a bit more.

The string table is a straightforward mechanism, but the prefixes
required some subtlety. They are encoded in a Pascal-string fashion with
a size followed by a sequence of offsets. This works relatively well for
the small realistic prefixes arrays in use.

Lots of code has to change in order to land this though: both all the
option library code has to be updated to use the string table and
prefixes table, and all the users of the options library have to be
updated to correctly instantiate the objects.

Some follow-up patches in the works to provide an abstraction for this
style of code, and to start using the same technique for some of the
other strings here now that the infrastructure is in place.

show more ...

[llvm][tools] Strip unneeded uses of raw_string_ostream::str() (NFC)

Remove unnecessary layer of indirection.

[llvm-driver] Fix usage of `InitLLVM` on Windows (#76306)

Previously, some tools such as `clang` or `lld` which require strict
order for certain command-line options, such as `clang -cc1` or `lld

[llvm-driver] Fix usage of `InitLLVM` on Windows (#76306)

Previously, some tools such as `clang` or `lld` which require strict
order for certain command-line options, such as `clang -cc1` or `lld
-flavor`, would not longer work on Windows, when these tools were linked
as part of `llvm-driver`. This was caused by `InitLLVM` which was part
of the `*_main()` function of these tools, which in turn calls
`windows::GetCommandLineArguments`. That function completly replaces
argc/argv by new UTF-8 contents, so any ajustements to argc/argv made by
`llvm-driver` prior to calling these tools was reset.

`InitLLVM` is now called by the `llvm-driver`. Any tool that
participates in (or is part of) the `llvm-driver` doesn't call
`InitLLVM` anymore.

show more ...

[llvm] Use StringRef::{starts,ends}_with (NFC) (#74956)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::

[llvm] Use StringRef::{starts,ends}_with (NFC) (#74956)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.

I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.

show more ...

[AIX] supporting -X options for llvm-ranlib in AIX OS

Summary:

llvm-ar is symlinked as llvm-ranlib and will act as ranlib when invoked in that mode. llvm-ar since [[ https://github.ibm.com/compiler

[AIX] supporting -X options for llvm-ranlib in AIX OS

Summary:

llvm-ar is symlinked as llvm-ranlib and will act as ranlib when invoked in that mode. llvm-ar since [[ https://github.ibm.com/compiler/llvm-project/commit/4f2cfbe5314b064625b2c87bde6ce5c8d04004c5 | compiler/llvm-project@4f2cfbe ]] supports the -X options, but doesn't seem to accept them when running as llvm-ranlib.

In AIX OS , according to https://www.ibm.com/docs/en/aix/7.2?topic=r-ranlib-command

-X mode Specifies the type of object file ranlib should examine. The mode must be one of the following:

32
Processes only 32-bit object files
64
Processes only 64-bit object files
32_64, any
Processes both 32-bit and 64-bit object files

The default is to process 32-bit object files (ignore 64-bit objects). The mode can also be set with the OBJECT_MODE environment variable. For example, OBJECT_MODE=64 causes ranlib to process any 64-bit objects and ignore 32-bit objects. The -X flag overrides the OBJECT_MODE variable.

Reviewers: James Henderson, MaskRay, Stephen Peckham
Differential Revision: https://reviews.llvm.org/D142660

show more ...

[llvm] Extract common `OptTable` bits into macros

All command-line tools using `llvm::opt` create an enum of option IDs and a table of `OptTable::Info` object. Most of the tools use the same ID (`OP

[llvm] Extract common `OptTable` bits into macros

All command-line tools using `llvm::opt` create an enum of option IDs and a table of `OptTable::Info` object. Most of the tools use the same ID (`OPT_##ID`), kind (`Option::KIND##Class`), group ID (`OPT_##GROUP`) and alias ID (`OPT_##ALIAS`). This patch extracts that common code into canonical macros. This results in fewer changes when tweaking the `OPTION` macros emitted by the TableGen backend.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D157028

show more ...

[dwp][libtool-darwin][sancov] Enable llvm-driver

Enable llvm-driver for:
llvm-dwp
llvm-libtoo-darwin
sancov

Differential Revision: https://reviews.llvm.org/D156758

[llvm-libtool-darwin] Use MapVector to avoid relying on StringMap iteration order

Simplify future changes like D142862 that change the hash function.

[llvm-libtool-darwin] Switch to OptTableSummary

Switch the parse of command line options fromllvm::cl to OptTable.
The motivation for this change is to continue adding llvm based tools
to the llvm d

[llvm-libtool-darwin] Switch to OptTableSummary

Switch the parse of command line options fromllvm::cl to OptTable.
The motivation for this change is to continue adding llvm based tools
to the llvm driver multicall.

Differential Revision: https://reviews.llvm.org/D153665

show more ...

[tools] llvm::Optional => std::optional

[tools] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of m

[tools] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

Reland "[llvm][NFC] Use c++17 style variable type traits"

This reverts commit 632a389f96355cbe7ed8fa7b8d2ed6267c92457c.

This relands commit
1834a310d060d55748ca38d4ae0482864c2047d8.

Differential R

Reland "[llvm][NFC] Use c++17 style variable type traits"

This reverts commit 632a389f96355cbe7ed8fa7b8d2ed6267c92457c.

This relands commit
1834a310d060d55748ca38d4ae0482864c2047d8.

Differential Revision: https://reviews.llvm.org/D137493

show more ...

Revert "[llvm][NFC] Use c++17 style variable type traits"

This reverts commit 1834a310d060d55748ca38d4ae0482864c2047d8.

[llvm][NFC] Use c++17 style variable type traits

This was done as a test for D137302 and it makes sense to push these changes

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/

[llvm][NFC] Use c++17 style variable type traits

This was done as a test for D137302 and it makes sense to push these changes

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D137493

show more ...

[llvm-libtool-darwin] Add support for -dependency_info

When using llvm-libtool-darwin as a drop in replacement for cctools
libtool, Xcode expects you to create a dependency info file. This file
is a

[llvm-libtool-darwin] Add support for -dependency_info

When using llvm-libtool-darwin as a drop in replacement for cctools
libtool, Xcode expects you to create a dependency info file. This file
is a very simple format describing the input files, the output files,
and the version of the tool. This logic is mirrored from that of
ld64.lld, which supports creating this file as well. Ideally we could
extract it, but I don't think we want to throw this into one of the
grab-bag libraries given how small the logic is.

Differential Revision: https://reviews.llvm.org/D134322

show more ...

[llvm-libtool-darwin] Parse and ignore -syslibroot

cctools libtool allows you to link dynamic libraries by passing through
a number of arguments to ld64. Because of this the default arguments
libtoo

[llvm-libtool-darwin] Parse and ignore -syslibroot

cctools libtool allows you to link dynamic libraries by passing through
a number of arguments to ld64. Because of this the default arguments
libtool receives in Xcode contains arguments that only matter in that
case. This change ignores this argument, at least until we ever support
that dynamic use case, so that you can use llvm-libtool-darwin as a
drop-in replacement in Xcode for cctools libtool. There are more
arguments we could ignore for this case, but we can probably add those
as the use case comes up.

Differential Revision: https://reviews.llvm.org/D134309

show more ...

Remove unneeded cl::ZeroOrMore for cl::opt/cl::list options

Remove unneeded cl::ZeroOrMore for cl::opt/cl::list options

Remove unneeded cl::ZeroOrMore for cl::opt options

Similar to 557efc9a8b68628c2c944678c6471dac30ed9e8e.
This commit handles options where cl::ZeroOrMore is more than one line below
cl::opt.

[llvm-libtools-darwin] Don't use global LLVMContext

Fixes some initialization order fiasco issues
https://lab.llvm.org/buildbot#builders/5/builds/20996

[llvm-libtool-darwin] Use cast<> instead of dyn_cast<> to avoid dereference of nullptr

The pointer is dereferenced immediately, so assert the cast is correct instead of returning nullptr

[llvm-libtool-darwin] Add -warnings_as_errors

libtool can currently produce 2 warnings:

1. No symbols were in the object file
2. An object file with the same basename was specified multiple times

[llvm-libtool-darwin] Add -warnings_as_errors

libtool can currently produce 2 warnings:

1. No symbols were in the object file
2. An object file with the same basename was specified multiple times

The first warning here is often harmless and may just mean you have some
translation units with no symbols for the target you're building for.
The second warning can lead to real issues like those mentioned in
https://reviews.llvm.org/D113130 where ODR violations can slip in.

This introduces a new -warnings_as_errors flag that can be used by build
systems that want to verify they never hit these warnings. For example
with bazel the libtool caller first uniques names to make sure the
duplicate base name case is not possible, but if that doesn't work as
expected, having it fail would be preferred.

It's also worth noting that llvm-libtool-darwin works around an issue
that cctools libtool experiences related to debug info and duplicate
basenames, the workaround is described here:
https://github.com/llvm/llvm-project/blob/30baa5d2a450d5e302d8cba3fc7a26a59d4b7ae1/llvm/lib/Object/ArchiveWriter.cpp#L424-L465
And it avoids this bug:
https://github.com/keith/radars/tree/f0cbbb1c37126ec6528c132510b29e08566377a7/DuplicateBasenameIssue

Differential Revision: https://reviews.llvm.org/D118931

show more ...

[llvm-libtool-darwin] Fix crash with bitcode asm module

When using llvm-libtool-darwin with LTO building llvm itself, it crashed
on a file with an asm module in the bitcode. This fixes that by
corre

[llvm-libtool-darwin] Fix crash with bitcode asm module

When using llvm-libtool-darwin with LTO building llvm itself, it crashed
on a file with an asm module in the bitcode. This fixes that by
correctly registering the targets for this.

Differential Revision: https://reviews.llvm.org/D118575

show more ...

[llvm-libtool-darwin] Remove var to fix use

This seems to have been moved so the second use is invalid on Linux but
not macOS

[llvm-libtool-darwin] Improve warning message for no symbols

This more closely mirrors apple's libtool, and also potentially makes it
clearer for multi-arch archives where the issue lies.

Different

[llvm-libtool-darwin] Improve warning message for no symbols

This more closely mirrors apple's libtool, and also potentially makes it
clearer for multi-arch archives where the issue lies.

Differential Revision: https://reviews.llvm.org/D118867

show more ...