[StrTable] Switch the option parser to `llvm::StringTable` (#123308)

Now that we have a dedicated abstraction for string tables, switch the
option parser library's string table over to it rather th

[StrTable] Switch the option parser to `llvm::StringTable` (#123308)

Now that we have a dedicated abstraction for string tables, switch the
option parser library's string table over to it rather than using a raw
`const char*`. Also try to use the `StringTable::Offset` type rather
than a raw `unsigned` where we can to avoid accidental increments or
other issues.

This is based on review feedback for the initial switch of options to a
string table. Happy to tweak or adjust if desired here.

show more ...

Rework the `Option` library to reduce dynamic relocations (#119198)

Apologies for the large change, I looked for ways to break this up and
all of the ones I saw added real complexity. This change f

Rework the `Option` library to reduce dynamic relocations (#119198)

Apologies for the large change, I looked for ways to break this up and
all of the ones I saw added real complexity. This change focuses on the
option's prefixed names and the array of prefixes. These are present in
every option and the dominant source of dynamic relocations for PIE or
PIC users of LLVM and Clang tooling. In some cases, 100s or 1000s of
them for the Clang driver which has a huge number of options.

This PR addresses this by building a string table and a prefixes table
that can be referenced with indices rather than pointers that require
dynamic relocations. This removes almost 7k dynmaic relocations from the
`clang` binary, roughly 8% of the remaining dynmaic relocations outside
of vtables. For busy-boxing use cases where many different option tables
are linked into the same binary, the savings add up a bit more.

The string table is a straightforward mechanism, but the prefixes
required some subtlety. They are encoded in a Pascal-string fashion with
a size followed by a sequence of offsets. This works relatively well for
the small realistic prefixes arrays in use.

Lots of code has to change in order to land this though: both all the
option library code has to be updated to use the string table and
prefixes table, and all the users of the options library have to be
updated to correctly instantiate the objects.

Some follow-up patches in the works to provide an abstraction for this
style of code, and to start using the same technique for some of the
other strings here now that the infrastructure is in place.

show more ...

[LLVM][TableGen] Rename Option emitter files (#109216)

Rename OptXXXEmitter.cpp to OptionXXXEmitter.cpp to have a less
ambiguous
name, as `Opt` could also mean optimization.

Reland "[flang][clang] Add Visibility specific help text for options (#81869)"

This reverts commit 67d20412b448533c77f54ac7a1e5d0775d273729.

This includes fixes for clanginstallapi.

Revert "[flang][clang] Add Visibility specific help text for options (#81869)"

This reverts commit 7e958f64efea6cb824e96ace51e021afbc150922.

Failing on multiple bots.

[flang][clang] Add Visibility specific help text for options (#81869)

And use it to print the correct default OpenMP version for flang and
flang -fc1.

This change adds an optional `HelpTextsForV

[flang][clang] Add Visibility specific help text for options (#81869)

And use it to print the correct default OpenMP version for flang and
flang -fc1.

This change adds an optional `HelpTextsForVariants` to options. This
allows you to change the help text that gets shown in documentation and
`--help` based on the program its being generated for.

As `OptTable` needs to be constexpr compatible, I have used a std::array
of help text variants. Each entry is:
(list of visibilities) - > help text string

So for the OpenMP version we have (flang, fc1) -> "OpenMP version for
flang is...".

So you can have multiple visibilities use the same string. The number of
entries is currently set to 1, and the number of visibilities per entry
is 2, because that's the maximum we need for now. The code is written so
we can increase these numbers later, and the unused elements will be initialised.

I have not applied this to group descriptions just because I don't know
of one that needs changing. It could easily be enabled for those too if
needed. There are minor changes to them just to get it all to compile.

This approach of storing many help strings per option in the 1 driver
library seemed preferable to making a whole new library for Flang (even
if that would mostly be including stuff from Clang).

show more ...

Reapply "[Option] Add "Visibility" field and clone the OptTable APIs to use it"

This reverts commit 4e3b89483a6922d3f48670bb1c50a37f342918c6, with
fixes for places I'd missed updating in lld and lld

Reapply "[Option] Add "Visibility" field and clone the OptTable APIs to use it"

This reverts commit 4e3b89483a6922d3f48670bb1c50a37f342918c6, with
fixes for places I'd missed updating in lld and lldb. I've also
renamed OptionVisibility::Default to "DefaultVis" to avoid ambiguity
since the undecorated name has to be available anywhere Options.inc is
included.

Original message follows:

This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and adds
variants of the OptTable APIs that use Visibility mask instead of
Include/Exclude flags.

We need to do this to clean up a bunch of complexity in the clang
driver's option handling - there's a whole slew of flags like
CoreOption, NoDriverOption, and FlangOnlyOption there today to try to
handle all of the permutations of flags that the various drivers need,
but it really doesn't scale well, as can be seen by things like the
somewhat recently introduced CLDXCOption.

Instead, we'll provide an additive model for visibility that's
separate from the other flags. For things like "HelpHidden", which is
used as a "subtractive" modifier for option visibility, we leave that
in "Flags" and handle it as a special case.

Note that we don't actually update the users of the Include/Exclude
APIs here or change the flags that exist in clang at all - that will
come in a follow up that refactors clang's Options.td to use the
increased flexibility this change allows.

Differential Revision: https://reviews.llvm.org/D157149

show more ...

Revert "[Option] Add "Visibility" field and clone the OptTable APIs to use it"

this is failing on bots, reverting to investigate.

This reverts commit a16104e6da6f36f3d72dbf53d10ba56495a0d65a.

[Option] Add "Visibility" field and clone the OptTable APIs to use it

This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and a

[Option] Add "Visibility" field and clone the OptTable APIs to use it

This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and adds
variants of the OptTable APIs that use Visibility mask instead of
Include/Exclude flags.

We need to do this to clean up a bunch of complexity in the clang
driver's option handling - there's a whole slew of flags like
CoreOption, NoDriverOption, and FlangOnlyOption there today to try to
handle all of the permutations of flags that the various drivers need,
but it really doesn't scale well, as can be seen by things like the
somewhat recently introduced CLDXCOption.

Instead, we'll provide an additive model for visibility that's
separate from the other flags. For things like "HelpHidden", which is
used as a "subtractive" modifier for option visibility, we leave that
in "Flags" and handle it as a special case.

Note that we don't actually update the users of the Include/Exclude
APIs here or change the flags that exist in clang at all - that will
come in a follow up that refactors clang's Options.td to use the
increased flexibility this change allows.

Differential Revision: https://reviews.llvm.org/D157149

show more ...

[llvm] Construct option's prefixed name at compile-time

Some Clang command-line handling code could benefit from the option's prefixed name being a `StringLiteral`. This patch changes the `llvm::opt

[llvm] Construct option's prefixed name at compile-time

Some Clang command-line handling code could benefit from the option's prefixed name being a `StringLiteral`. This patch changes the `llvm::opt` TableGen backend to generate and emit that into the .inc file.

Depends on D157028.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D157029

show more ...

Recommit of 8ae18303f97d5dcfaecc90b4d87effb2011ed82e - part 2

Differential Revision: https://reviews.llvm.org/D139274

Revert "Recommit of 8ae18303f97d5dcfaecc90b4d87effb2011ed82e - part 2"

This reverts commit 4faf00006cf989f3ae212912994022c0486a2dc4.

Recommit of 8ae18303f97d5dcfaecc90b4d87effb2011ed82e - part 2

Differential Revision: https://reviews.llvm.org/D139274

Revert "Store OptTable::Info::Name as a StringRef"

Another revert, for another set of issues I don't reproduce locally...

see https://lab.llvm.org/buildbot/#/builders/139/builds/32327

This reverts

Revert "Store OptTable::Info::Name as a StringRef"

Another revert, for another set of issues I don't reproduce locally...

see https://lab.llvm.org/buildbot/#/builders/139/builds/32327

This reverts commit bdfa3100dc3ea9e9ce4d3d4100ea6bb4c3fa2b81.

show more ...

Store OptTable::Info::Name as a StringRef

This is a recommit of 8ae18303f97d5dcfaecc90b4d87effb2011ed82e,
with a few cleanups.

This avoids implicit conversion to StringRef at several points, which

Store OptTable::Info::Name as a StringRef

This is a recommit of 8ae18303f97d5dcfaecc90b4d87effb2011ed82e,
with a few cleanups.

This avoids implicit conversion to StringRef at several points, which in
turns avoid redundant calls to strlen.

As a side effect, this greatly simplifies the implementation of
StrCmpOptionNameIgnoreCase.

It also eventually gives a consistent, humble speedup in compilation
time (timing updated since original commit).

https://llvm-compile-time-tracker.com/compare.php?from=76fcfea283472a80356d87c89270b0e2d106b54c&to=b70eb1f347f22fe4d2977360c4ed701eabc43994&stat=instructions:u

Differential Revision: https://reviews.llvm.org/D139274

show more ...

Revert "Store OptTable::Info::Name as a StringRef"

This reverts commit 5951b0bb23f3265bea16f28c2af9d278b9d829c6.

This is causing 24 test failures on the PS4 linux bot: https://lab.llvm.org/buildbot

Revert "Store OptTable::Info::Name as a StringRef"

This reverts commit 5951b0bb23f3265bea16f28c2af9d278b9d829c6.

This is causing 24 test failures on the PS4 linux bot: https://lab.llvm.org/buildbot/#/builders/139/builds/32263

show more ...

Store OptTable::Info::Name as a StringRef

This is a recommit of 8ae18303f97d5dcfaecc90b4d87effb2011ed82e,
with a few cleanups.

This avoids implicit conversion to StringRef at several points, which

Store OptTable::Info::Name as a StringRef

This is a recommit of 8ae18303f97d5dcfaecc90b4d87effb2011ed82e,
with a few cleanups.

This avoids implicit conversion to StringRef at several points, which in
turns avoid redundant calls to strlen.

As a side effect, this greatly simplifies the implementation of
StrCmpOptionNameIgnoreCase.

It also eventually gives a consistent, humble speedup in compilation
time (timing updated since original commit).

https://llvm-compile-time-tracker.com/compare.php?from=de4b6a1bc64db33643f001ad45fae7b92b4a4688&to=c23a93d1292052b4be2fbe8c586fa31143d0c7ed&stat=instructions:u

Differential Revision: https://reviews.llvm.org/D139274

show more ...

Revert "Store OptTable::Info::Name as a StringRef"

This reverts commit 8ae18303f97d5dcfaecc90b4d87effb2011ed82e.
Breaks buildling lldb, see https://reviews.llvm.org/D139274#3974171

Store OptTable::Info::Name as a StringRef

This avoids implicit conversion to StringRef at several points, which in
turns avoid redundant calls to strlen.

As a side effect, this greatly simplifies t

Store OptTable::Info::Name as a StringRef

This avoids implicit conversion to StringRef at several points, which in
turns avoid redundant calls to strlen.

As a side effect, this greatly simplifies the implementation of
StrCmpOptionNameIgnoreCase.

It also eventually gives a consistent, humble speedup in compilation
time.

https://llvm-compile-time-tracker.com/compare.php?from=5f5b942823474e98e43a27d515a87ce140396c53&to=60e13b778119fc32d50dc38ff1a564a87146e9c6&stat=instructions:u

Differential Revision: https://reviews.llvm.org/D139274

show more ...

[clang][cli] Specify KeyPath prefixes via TableGen classes

It turns out we need to handle `LangOptions` separately from the rest of the options. `LangOptions` used to be conditionally parsed only wh

[clang][cli] Specify KeyPath prefixes via TableGen classes

It turns out we need to handle `LangOptions` separately from the rest of the options. `LangOptions` used to be conditionally parsed only when `!(DashX.getFormat() == InputKind::Precompiled || DashX.getLanguage() == Language::LLVM_IR)` and we need to restore this order (for more info, see D94682).

We could do this similarly to how `DiagnosticOptions` are handled: via a counterpart to the `IsDiag` mix-in (e.g. `IsLang`). These mix-ins would prefix the option key path with the appropriate `CompilerInvocation::XxxOpts` member. However, this solution would be problematic, as we'd now have two kinds of options (`Lang` and `Diag`) with seemingly incomplete key paths in the same file. To understand what `CompilerInvocation` member an option affects, one would need to read the whole option definition and notice the `IsDiag` or `IsLang` class.

Instead, this patch introduces more robust way to handle different kinds of options separately: via the `KeyPathAndMacroPrefix` class. We have one specialization of that class per `CompilerInvocation` member (e.g. `LangOpts`, `DiagnosticOpts`, etc.). Now, instead of specifying a key path with `"LangOpts->UndefPrefixes"`, we use `LangOpts<"UndefPrefixes">`. This keeps the readability intact (you don't have to look for the `IsLang` mix-in, the key path is complete on its own) and allows us to specify a custom macro prefix within `LangOpts`.

Reviewed By: Bigcheese

Differential Revision: https://reviews.llvm.org/D94676

show more ...

Reapply "[clang][cli] Allow users to specify a conditional to prevent parsing options with MarshallingInfo"

This reverts commit d0fa7a05 and fixes failing OptionMarshallingTest by adding the SHOULD_

Reapply "[clang][cli] Allow users to specify a conditional to prevent parsing options with MarshallingInfo"

This reverts commit d0fa7a05 and fixes failing OptionMarshallingTest by adding the SHOULD_PARSE macro argument

show more ...

[clang][cli] Split DefaultAnyOf into a default value and ImpliedByAnyOf

This makes the options API composable, allows boolean flags to imply non-boolean values and makes the code more logical (IMO).

[clang][cli] Split DefaultAnyOf into a default value and ImpliedByAnyOf

This makes the options API composable, allows boolean flags to imply non-boolean values and makes the code more logical (IMO).

Differential Revision: https://reviews.llvm.org/D91861

show more ...

[clang][cli] Remove NormalizerRetTy and use the decltype of the KeyPath instead

Depends on D83315

Reviewed By: Bigcheese

Original patch by Daniel Grumberg.

Differential Revision: https://reviews.

[clang][cli] Remove NormalizerRetTy and use the decltype of the KeyPath instead

Depends on D83315

Reviewed By: Bigcheese

Original patch by Daniel Grumberg.

Differential Revision: https://reviews.llvm.org/D83406

show more ...

Reland [clang][cli] Port ObjCMTAction to new option parsing system

Merge existing marhsalling info kinds and add some primitives to
express flag options that contribute to a bitfield.

Depends on D8

Reland [clang][cli] Port ObjCMTAction to new option parsing system

Merge existing marhsalling info kinds and add some primitives to
express flag options that contribute to a bitfield.

Depends on D82574

Original patch by Daniel Grumberg.

Reviewed By: Bigcheese

Differential Revision: https://reviews.llvm.org/D82860

show more ...

Revert "[clang][cli] Port ObjCMTAction to new option parsing system"

This reverts commit 09248a5d25bb1c9f357247fa3da8fbe4470e9c67.

Some builds are broken. I suspect a `static constexpr` in a class

Revert "[clang][cli] Port ObjCMTAction to new option parsing system"

This reverts commit 09248a5d25bb1c9f357247fa3da8fbe4470e9c67.

Some builds are broken. I suspect a `static constexpr` in a class missing a
definition out of class (required pre-c++17).

show more ...