[llvm-lib] Don't rewrite paths for members in non-thin archives (#123416)

This matches what MS lib.exe does (and llvm-ar too); when adding files
to an archive, MS lib.exe stores the file name as it

[llvm-lib] Don't rewrite paths for members in non-thin archives (#123416)

This matches what MS lib.exe does (and llvm-ar too); when adding files
to an archive, MS lib.exe stores the file name as it was given on the
command line, whereas llvm-lib rewrote it into a relative path name,
relative to the archive location. Such a rewrite makes sense for thin
archives, but not for regular archives. (MS lib.exe doesn't support
producing thin archives; that's an LLVM extension - see the
thin-relative.test testcase.)

The behaviour to rewrite these paths was added in
451c2ef199e9c5163007ac32e2d426fbfb37e664; it is unclear why it was
chosen to do the rewriting for non-thin archives as well. This quirk is
even pointed out in a code comment - but neither the code review at
https://reviews.llvm.org/D57842 nor the linked bug report at
https://crbug.com/41440160 mentions why this is done for all archives,
not only thin ones.

Therefore, assume that this only was done out of convenience, and change
llvm-lib to not adjust the paths for non-thin archives.

Normally, the actual member names doesn't matter for non-thin archives;
however for short import libraries, where each member is named e.g.
"foo.dll", the names do matter. If using llvm-lib to merge two import
libraries (as a non-thin library), preserve the original names rather
than making the member names relative.

show more ...

[llvm-lib] Handle MIPS architecture (#121254)

- add a test to check values for /machine argument
- add a test to check if machine is correctly inferred from inputs

Revert "[llvm-lib] Handle MIPS architecture (#121007)"

This reverts commit 5d529c32cc2d5342a0d183881b6c3023435ed5d3.

[llvm-lib] Handle MIPS architecture (#121007)

- add a test to check values for /machine argument
- add a test to check if machine is correctly inferred from inputs

[llvm-lib] Add /llvmlibindex:no to disable writing an index (#120596)

This can be used with /llvmlibthin to create thin archives without an
index, which is a prerequisite for porting
https://revie

[llvm-lib] Add /llvmlibindex:no to disable writing an index (#120596)

This can be used with /llvmlibthin to create thin archives without an
index, which is a prerequisite for porting
https://reviews.llvm.org/D117284 to lld-link.

Creating files like this is already possible with `llvm-ar rcS`, so this
doesn't add additional problems.

show more ...

Rework the `Option` library to reduce dynamic relocations (#119198)

Apologies for the large change, I looked for ways to break this up and
all of the ones I saw added real complexity. This change f

Rework the `Option` library to reduce dynamic relocations (#119198)

Apologies for the large change, I looked for ways to break this up and
all of the ones I saw added real complexity. This change focuses on the
option's prefixed names and the array of prefixes. These are present in
every option and the dominant source of dynamic relocations for PIE or
PIC users of LLVM and Clang tooling. In some cases, 100s or 1000s of
them for the Clang driver which has a huge number of options.

This PR addresses this by building a string table and a prefixes table
that can be referenced with indices rather than pointers that require
dynamic relocations. This removes almost 7k dynmaic relocations from the
`clang` binary, roughly 8% of the remaining dynmaic relocations outside
of vtables. For busy-boxing use cases where many different option tables
are linked into the same binary, the savings add up a bit more.

The string table is a straightforward mechanism, but the prefixes
required some subtlety. They are encoded in a Pascal-string fashion with
a size followed by a sequence of offsets. This works relatively well for
the small realistic prefixes arrays in use.

Lots of code has to change in order to land this though: both all the
option library code has to be updated to use the string table and
prefixes table, and all the users of the options library have to be
updated to correctly instantiate the objects.

Some follow-up patches in the works to provide an abstraction for this
style of code, and to start using the same technique for some of the
other strings here now that the infrastructure is in place.

show more ...

[llvm-lib][llvm-dlltool] Fix handling of invalid ARM64EC function names (#116250)

This is a follow-up to #115567. Emit an error for invalid function
names, similar to MSVC's `lib.exe` behavior.

[llvm-lib][llvm-dlltool] Fix handling of invalid ARM64EC function names (#116250)

This is a follow-up to #115567. Emit an error for invalid function
names, similar to MSVC's `lib.exe` behavior.

Returning an error from `writeImportLibrary` exposed bugs in error
handling by its callers, which have been addressed in this patch.

show more ...

[llvm] Construct SmallVector with ArrayRef (NFC) (#101872)

[llvm] Remove redundant calls to std::unique_ptr<T>::get (NFC) (#97914)

[z/OS] treat text files as text files so auto-conversion is done (#90128)

To support auto-conversion on z/OS text files need to be opened as text files. These changes will fix a number of LIT failur

[z/OS] treat text files as text files so auto-conversion is done (#90128)

To support auto-conversion on z/OS text files need to be opened as text files. These changes will fix a number of LIT failures due to text files not being converted to the internal code page.

update a number of tools so they open the text files as text files
add support in the cat.py to open a text file as a text file (Windows will continue to treat all files as binary so new lines are handled correctly)
add env var definitions to enable auto-conversion in the lit config file.

show more ...

[Object][COFF][NFC] Make writeImportLibrary NativeExports argument optional. (#81600)

It's not interesting for majority of downstream users.

[llvm-lib] Add support for -defArm64Native argument. (#81426)

This can be used to create import libraries that contain both ARM64EC
and native exports. The implementation follows observed MSVC lib.

[llvm-lib] Add support for -defArm64Native argument. (#81426)

This can be used to create import libraries that contain both ARM64EC
and native exports. The implementation follows observed MSVC lib.exe
behaviour. It's ignored on targets other than ARM64EC.

show more ...

[llvm] Use SmallString::operator std::string (NFC)

[AIX] supporting -X options for llvm-ranlib in AIX OS

Summary:

llvm-ar is symlinked as llvm-ranlib and will act as ranlib when invoked in that mode. llvm-ar since [[ https://github.ibm.com/compiler

[AIX] supporting -X options for llvm-ranlib in AIX OS

Summary:

llvm-ar is symlinked as llvm-ranlib and will act as ranlib when invoked in that mode. llvm-ar since [[ https://github.ibm.com/compiler/llvm-project/commit/4f2cfbe5314b064625b2c87bde6ce5c8d04004c5 | compiler/llvm-project@4f2cfbe ]] supports the -X options, but doesn't seem to accept them when running as llvm-ranlib.

In AIX OS , according to https://www.ibm.com/docs/en/aix/7.2?topic=r-ranlib-command

-X mode Specifies the type of object file ranlib should examine. The mode must be one of the following:

32
Processes only 32-bit object files
64
Processes only 64-bit object files
32_64, any
Processes both 32-bit and 64-bit object files

The default is to process 32-bit object files (ignore 64-bit objects). The mode can also be set with the OBJECT_MODE environment variable. For example, OBJECT_MODE=64 causes ranlib to process any 64-bit objects and ignore 32-bit objects. The -X flag overrides the OBJECT_MODE variable.

Reviewers: James Henderson, MaskRay, Stephen Peckham
Differential Revision: https://reviews.llvm.org/D142660

show more ...

Reapply "[Option] Add "Visibility" field and clone the OptTable APIs to use it"

This reverts commit 4e3b89483a6922d3f48670bb1c50a37f342918c6, with
fixes for places I'd missed updating in lld and lld

Reapply "[Option] Add "Visibility" field and clone the OptTable APIs to use it"

This reverts commit 4e3b89483a6922d3f48670bb1c50a37f342918c6, with
fixes for places I'd missed updating in lld and lldb. I've also
renamed OptionVisibility::Default to "DefaultVis" to avoid ambiguity
since the undecorated name has to be available anywhere Options.inc is
included.

Original message follows:

This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and adds
variants of the OptTable APIs that use Visibility mask instead of
Include/Exclude flags.

We need to do this to clean up a bunch of complexity in the clang
driver's option handling - there's a whole slew of flags like
CoreOption, NoDriverOption, and FlangOnlyOption there today to try to
handle all of the permutations of flags that the various drivers need,
but it really doesn't scale well, as can be seen by things like the
somewhat recently introduced CLDXCOption.

Instead, we'll provide an additive model for visibility that's
separate from the other flags. For things like "HelpHidden", which is
used as a "subtractive" modifier for option visibility, we leave that
in "Flags" and handle it as a special case.

Note that we don't actually update the users of the Include/Exclude
APIs here or change the flags that exist in clang at all - that will
come in a follow up that refactors clang's Options.td to use the
increased flexibility this change allows.

Differential Revision: https://reviews.llvm.org/D157149

show more ...

Revert "[Option] Add "Visibility" field and clone the OptTable APIs to use it"

this is failing on bots, reverting to investigate.

This reverts commit a16104e6da6f36f3d72dbf53d10ba56495a0d65a.

[Option] Add "Visibility" field and clone the OptTable APIs to use it

This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and a

[Option] Add "Visibility" field and clone the OptTable APIs to use it

This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and adds
variants of the OptTable APIs that use Visibility mask instead of
Include/Exclude flags.

We need to do this to clean up a bunch of complexity in the clang
driver's option handling - there's a whole slew of flags like
CoreOption, NoDriverOption, and FlangOnlyOption there today to try to
handle all of the permutations of flags that the various drivers need,
but it really doesn't scale well, as can be seen by things like the
somewhat recently introduced CLDXCOption.

Instead, we'll provide an additive model for visibility that's
separate from the other flags. For things like "HelpHidden", which is
used as a "subtractive" modifier for option visibility, we leave that
in "Flags" and handle it as a special case.

Note that we don't actually update the users of the Include/Exclude
APIs here or change the flags that exist in clang at all - that will
come in a follow up that refactors clang's Options.td to use the
increased flexibility this change allows.

Differential Revision: https://reviews.llvm.org/D157149

show more ...

[llvm] Extract common `OptTable` bits into macros

All command-line tools using `llvm::opt` create an enum of option IDs and a table of `OptTable::Info` object. Most of the tools use the same ID (`OP

[llvm] Extract common `OptTable` bits into macros

All command-line tools using `llvm::opt` create an enum of option IDs and a table of `OptTable::Info` object. Most of the tools use the same ID (`OPT_##ID`), kind (`Option::KIND##Class`), group ID (`OPT_##GROUP`) and alias ID (`OPT_##ALIAS`). This patch extracts that common code into canonical macros. This results in fewer changes when tweaking the `OPTION` macros emitted by the TableGen backend.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D157028

show more ...

[llvm-lib] Pass the right value of MingwDef to parseCOFFModuleDefinition

In llvm-lib cases, the MinGW specific behaviours shouldn't be
enabled.

Not adding any tests to check the corner case behavio

[llvm-lib] Pass the right value of MingwDef to parseCOFFModuleDefinition

In llvm-lib cases, the MinGW specific behaviours shouldn't be
enabled.

Not adding any tests to check the corner case behaviours, as this
simply was a mistake when this codepath was added in
395ec4458fb7fc700551f7017c0a395d68c55873 / D144765.

Differential Revision: https://reviews.llvm.org/D152362

show more ...

[llvm-lib] [llvm-readobj] [llvm-cvtres] Add Support for ARM64X object files.

Similar to D125411, but for ARM64X.

ARM64X PE binaries are hybrids containing both ARM64EC and pure ARM64
variants in on

[llvm-lib] [llvm-readobj] [llvm-cvtres] Add Support for ARM64X object files.

Similar to D125411, but for ARM64X.

ARM64X PE binaries are hybrids containing both ARM64EC and pure ARM64
variants in one file. They are usually linked by passing separate
ARM64EC and ARM64 object files to linker. Linked binaries use ARM64
machine and contain additional CHPE metadata in their load config.
CHPE metadata support is not part of this patch, I plan to send that later.

Using ARM64X as a machine type of object files themselves is somewhat
ambiguous, but such files are allowed by MSVC. It treats them as ARM64
or ARM64EC object, depending on the context. Such objects can be
produced with cvtres.exe -machine:arm64x.

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D148517

show more ...

[llvm-lib] Add support for ARM64EC libraries.

ARM64EC allows having both pure ARM64 objects and ARM64EC in the
same archive. This allows using single static library for linking
pure ARM64, pure ARM6

[llvm-lib] Add support for ARM64EC libraries.

ARM64EC allows having both pure ARM64 objects and ARM64EC in the
same archive. This allows using single static library for linking
pure ARM64, pure ARM64EC or mixed modules (what MS calls ARM64X:
a single module that may be used in both modes). To achieve that,
such static libraries need two separated symbol maps. The usual map
contains only pure ARM64 symbols, while a new /<ECSYMBOLS>/ section
contains EC symbols. EC symbols map has very similar format to the
usual map, except it doesn't contain object offsets and uses offsets
from regular map instead. This is true even for pure ARM64EC static
library: it will simply have 0 symbols in the symbol map.

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D143541

show more ...

[llvm-lib] Use COFF archive format in llvm-lib (other archive tools don't use this format).

We currently just use GNU format for llvm-lib. This mostly works, but
ARM64EC needs an additional section

[llvm-lib] Use COFF archive format in llvm-lib (other archive tools don't use this format).

We currently just use GNU format for llvm-lib. This mostly works, but
ARM64EC needs an additional section that does not really fit GNU format.
This patch implements writing in COFF format (as in, it's what archive
reader considers as K_COFF). This mostly requires symbol emitting symbol
map. Note that, just like in case of MSVC, symbols are de-duplicated in
both usual symbol table and the new symbol map.

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D143540

show more ...

[llvm-lib] 'llvm-lib' currently cannot generate an import library from a Windows
.def file, functionality supported by 'lib'. This incompatibility is
breaking clang based Windows openmp builds. This

[llvm-lib] 'llvm-lib' currently cannot generate an import library from a Windows
.def file, functionality supported by 'lib'. This incompatibility is
breaking clang based Windows openmp builds. This revision adds
basic support for this feature to llvm-lib by cloning the corresponding
code from 'dlltool'.

Differential Revision:https://reviews.llvm.org/D144765

show more ...

[llvm-lib] Write object files in reversed order.

This isn't strictly needed, but this matches how MSVC lib.exe writes to
archives, so this makes llvm-lib more compatible and simplifies comparing
out

[llvm-lib] Write object files in reversed order.

This isn't strictly needed, but this matches how MSVC lib.exe writes to
archives, so this makes llvm-lib more compatible and simplifies comparing
output between tools.

Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D143536

show more ...

[OptTable] Precompute OptTable prefixes union table through tablegen

This avoid rediscovering this table when reading each options, providing
a sensible 2% speedup when processing and empty file, an

[OptTable] Precompute OptTable prefixes union table through tablegen

This avoid rediscovering this table when reading each options, providing
a sensible 2% speedup when processing and empty file, and a measurable
speedup on typical workloads, see:

This is optional, the legacy, on-the-fly, approach can still be used
through the GenericOptTable class, while the new one is used through
PrecomputedOptTable.

https://llvm-compile-time-tracker.com/compare.php?from=4da6cb3202817ee2897d6b690e4af950459caea4&to=19a492b704e8f5c1dea120b9c0d3859bd78796be&stat=instructions:u

Differential Revision: https://reviews.llvm.org/D140800

show more ...