[flang][driver] rename flang-new to flang (#110023)

This does a global rename from `flang-new` to `flang`. I also
removed/changed any TODOs that I found related to making this change.

---------

[flang][driver] rename flang-new to flang (#110023)

This does a global rename from `flang-new` to `flang`. I also
removed/changed any TODOs that I found related to making this change.

---------

Co-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Andrzej Warzynski <andrzej.warzynski@arm.com>

show more ...

[flang] New -fdebug-unparse-with-modules option (#91660)

This option is a compilation action that parses a source file and
performs semantic analysis on it, like the existing -fdebug-unparse
optio

[flang] New -fdebug-unparse-with-modules option (#91660)

This option is a compilation action that parses a source file and
performs semantic analysis on it, like the existing -fdebug-unparse
option does. Its output, however, is preceded by the effective contents
of all of the non-intrinsic modules on which it depends but does not
define, transitively preceded by the closure of all of those modules'
dependencies.

The output from this option is therefore the analyzed parse tree for a
source file encapsulated with all of its non-intrinsic module
dependencies. This output may be useful for extracting code from large
applications for use as an attachment to a bug report, or as input to a
test case reduction tool for problem isolation.

show more ...

[flang] Use StringRef::{starts,ends}_with (NFC)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,en

[flang] Use StringRef::{starts,ends}_with (NFC)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.

I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.

show more ...

[Flang][Driver] Add warning support for invalid R_Group options

With the R_Group options, invalid values e.g. '-Rpa' will not emit
a warning like clang. This patch enables warning reporting, as
well

[Flang][Driver] Add warning support for invalid R_Group options

With the R_Group options, invalid values e.g. '-Rpa' will not emit
a warning like clang. This patch enables warning reporting, as
well as suggestions on what option the user intended to select.

Depends on D158174 and D158436. The former, adds backend
support to R_Group options while the latter, implements
regex support with some tests refactoring that cause a merge
conflict.

Reviewed By: awarzynski

Differential Revision: https://reviews.llvm.org/D158593

show more ...

[Flang][Driver] Add regex support for R_Group options

Add regex handling for all variations of OPT_R_Joined, i.e.
`-Rpass`, `-Rpass-analysis`, `-Rpass-missed`.

Depends on D158174. That patch implem

[Flang][Driver] Add regex support for R_Group options

Add regex handling for all variations of OPT_R_Joined, i.e.
`-Rpass`, `-Rpass-analysis`, `-Rpass-missed`.

Depends on D158174. That patch implements backend support for
R_Group options.

Reviewed By: awarzynski

Differential Revision: https://reviews.llvm.org/D158436

show more ...

[Flang][Driver] Implement OPT_R_Joined options

Add a BackendRemarkConsumer class, responsible for handling diagnostics
received from LLVM. The diagnostics being information on middle and
backend pas

[Flang][Driver] Implement OPT_R_Joined options

Add a BackendRemarkConsumer class, responsible for handling diagnostics
received from LLVM. The diagnostics being information on middle and
backend passes used or not used.

Clang by default has all remarks ignored but manually sets the severity of
`R_Group` to visible(`clang::diag::clang::Severity::Remark`). This patch does
the same for Flang.

Depends on D157410. That patch adds the R family of options to
`FlangOption` and `FC1Option` in
`clang/include/clang/Driver/Options.td`

Reviewed By: awarzynski

Differential Revision: https://reviews.llvm.org/D158174

show more ...

[Driver] Refactor to use llvm Option's new Visibility flags

This is a big refactor of the clang driver's option handling to use
the Visibility flags introduced in https://reviews.llvm.org/D157149.
T

[Driver] Refactor to use llvm Option's new Visibility flags

This is a big refactor of the clang driver's option handling to use
the Visibility flags introduced in https://reviews.llvm.org/D157149.
There are a few distinct parts, but they can't really be split into
separate commits and still be made to compile.

1. We split out some of the flags in ClangFlags to ClangVisibility.
Note that this does not include any subtractive flags.

2. We update the Flag definitions and OptIn/OptOut constructs in
Options.td by hand.

3. We introduce and use a script, update_options_td_flags, to ease
migration of flag definitions in Options.td, and we run that on
Options.td. I intend to remove this later, but I'm committing it so
that downstream forks can use the script to simplify merging.

4. We update calls to OptTable in the clang driver, cc1as, flang, and
clangd to use the visibility APIs instead of Include/Exclude flags.

5. We deprecate the Include/Exclude APIs and add a release note.

*if you are running into conflicts with this change:*

Note that https://reviews.llvm.org/D157150 may also be the culprit and
if so it should be handled first.

The script in `clang/utils/update_options_td_flags.py` can help. Take
the downstream side of all conflicts and then run the following:

```
% cd clang/include/clang/Driver
% ../../../utils/update_options_td_flags.py Options.td > Options.td.new
% mv Options.td.new Options.td
```

This will hopefully be sufficient, please take a look at the diff.

Differential Revision: https://reviews.llvm.org/D157151

show more ...

[flang][hlfir] Separate -emit-fir and -emit-hlfir for flang-new

In review for https://reviews.llvm.org/D146278, @vzakhari asked to
separate -emit-fir and -emit-hlfir. This will allow FIR to be easil

[flang][hlfir] Separate -emit-fir and -emit-hlfir for flang-new

In review for https://reviews.llvm.org/D146278, @vzakhari asked to
separate -emit-fir and -emit-hlfir. This will allow FIR to be easily
outputted after the HLFIR passes have been run.

The new semantics are as follows:

| Action | -flang-experimental-hlfir? | Result |
| =========== | ========================== | =============================== |
| -emit-hlfir | N | Outputs HLFIR |
| -emit-hlfir | Y | Outputs HLFIR |
| -emit-fir | N | Outputs FIR, using old lowering |
| -emit-fir | Y | Outputs FIR, lowering via HLFIR |

A patch for bbc will follow.

Differential Revision: https://reviews.llvm.org/D151088

show more ...

[Flang] Run clang-format on all flang files

This will make it easier for me to do reviews.

Differential Revision: https://reviews.llvm.org/D137291

[Flang][Driver] Add support for AsmPrinter -mmlir options

This patch adds support for AsmPrinter `-mmlir` options to the Flang driver.

Reviewed By: awarzynski

Differential Revision: https://revie

[Flang][Driver] Add support for AsmPrinter -mmlir options

This patch adds support for AsmPrinter `-mmlir` options to the Flang driver.

Reviewed By: awarzynski

Differential Revision: https://reviews.llvm.org/D130598

show more ...

[flang][Driver] Refine _when_ driver diagnostics are formatted

This patch refines //when// driver diagnostics are formatted so that
`flang-new` and `flang-new -fc1` behave consistently with `clang`

[flang][Driver] Refine _when_ driver diagnostics are formatted

This patch refines //when// driver diagnostics are formatted so that
`flang-new` and `flang-new -fc1` behave consistently with `clang` and
`clang -cc1`, respectively. This change only applies to driver diagnostics.
Scanning, parsing and semantic diagnostics are separate and not covered here.

**NEW BEHAVIOUR**
To illustrate the new behaviour, consider the following input file:
```! file.f90
program m
integer :: i = k
end
```
In the following invocations, "error: Semantic errors in file.f90" _will be_
formatted:
```
$ flang-new file.f90
error: Semantic errors in file.f90
./file.f90:2:18: error: Must be a constant value
integer :: i = k
$ flang-new -fc1 -fcolor-diagnostics file.f90
error: Semantic errors in file.f90
./file.f90:2:18: error: Must be a constant value
integer :: i = k
```

However, in the following invocations, "error: Semantic errors in file.f90"
_will not be_ formatted:
```
$ flang-new -fno-color-diagnostics file.f90
error: Semantic errors in file.f90
./file.f90:2:18: error: Must be a constant value
integer :: i = k
$ flang-new -fc1 file.f90
error: Semantic errors in file.f90
./file.f90:2:18: error: Must be a constant value
integer :: i = k
```

Before this change, none of the above would be formatted. Note also that the
default behaviour in `flang-new` is different to `flang-new -fc1` (this is
consistent with Clang).

**NOTES ON IMPLEMENTATION**
Note that the diagnostic options are parsed in `createAndPopulateDiagOpt`s in
driver.cpp. That's where the driver's `DiagnosticEngine` options are set. Like
most command-line compiler driver options, these flags are "claimed" in
Flang.cpp (i.e. when creating a frontend driver invocation) by calling
`getLastArg` rather than in driver.cpp.

In Clang's Options.td, `defm color_diagnostics` is replaced with two separate
definitions: `def fcolor_diagnostics` and def fno_color_diagnostics`. That's
because originally `color_diagnostics` derived from `OptInCC1FFlag`, which is a
multiclass for opt-in options in CC1. In order to preserve the current
behaviour in `clang -cc1` (i.e. to keep `-fno-color-diagnostics` unavailable in
`clang -cc1`) and to implement similar behaviour in `flang-new -fc1`, we can't
re-use `OptInCC1FFlag`.

Formatting is only available in consoles that support it and will normally mean that
the message is printed in bold + color.

Co-authored-by: Andrzej Warzynski <andrzej.warzynski@arm.com>

Reviewed By: rovka

Differential Revision: https://reviews.llvm.org/D126164

show more ...

[flang][driver] Switch to the MLIR coding style in the driver (nfc)

This patch re-factors the driver code in LLVM Flang (frontend +
compiler) to use the MLIR style. For more context, please see:
htt

[flang][driver] Switch to the MLIR coding style in the driver (nfc)

This patch re-factors the driver code in LLVM Flang (frontend +
compiler) to use the MLIR style. For more context, please see:
https://discourse.llvm.org/t/rfc-coding-style-in-the-driver/

Most changes here are rather self-explanatory. Accessors are renamed to
be more consistent with the rest of LLVM (e.g. allSource -->
getAllSources). Additionally, MLIR clang-tidy files are added in the
affected directories.

clang-tidy and clang-format files were copied from MLIR. Small
additional changes are made to silence clang-tidy/clang-format
warnings.

[1] https://mlir.llvm.org/getting_started/DeveloperGuide/

Differential Revision: https://reviews.llvm.org/D125007

show more ...

[flang][driver] Re-organise the code-gen actions (nfc)

All frontend actions that generate code (MLIR, LLVM IR/BC,
Assembly/Object Code) are re-factored as essentially one action,
`CodeGenAction`, w

[flang][driver] Re-organise the code-gen actions (nfc)

All frontend actions that generate code (MLIR, LLVM IR/BC,
Assembly/Object Code) are re-factored as essentially one action,
`CodeGenAction`, with minor specialisations. To facilate all this,
`CodeGenAction` is extended to hold `TargetMachine` and backend action
type (MLIR vs LLVM IR vs LLVM BC vs Assembly vs Object Code).

`CodeGenAction` is no longer a pure abstract class and the
corresponding `ExecuteAction` is implemented so that it covers all use
cases. All this allows a much better code re-use.

Key functionality is extracted into some helpful hooks:
* `SetUpTargetMachine`
* `GetOutputStream`
* `EmitObjectCodeHelper`
* `EmitBCHelper`
I hope that this clarifies the overall structure. I suspect that we may
need to revisit this again as the functionality grows in complexity.

Differential Revision: https://reviews.llvm.org/D124665

show more ...

[flang][driver] NFC: Make code more in line with LLVM style

This patch basically implements [1] in ExecuteCompilerInvocation.cpp. It
also:
* replaces `CreateFrontendBaseAction` with `CreateFronten

[flang][driver] NFC: Make code more in line with LLVM style

This patch basically implements [1] in ExecuteCompilerInvocation.cpp. It
also:
* replaces `CreateFrontendBaseAction` with `CreateFrontendAction`
(only one method is needed ATM, this change removes the extra
indirection)
* removes `InvalidAction` from the `ActionKind` enum (I don't think it
adds much and keeping it would mean adding a new void case in
`CreateFrontendAction`)
* sets the default frontend action in FrontendOptions.h to
`ParseSyntaxOnly` (note that this is still overridden independently
in `ParseFrontendArg` in CompilerInvocation.cpp)

No new functionality is added, hence no tests.

[1] https://llvm.org/docs/CodingStandards.html#don-t-use-default-labels-in-fully-covered-switches-over-enumerations

Differential Revision: https://reviews.llvm.org/D124245

show more ...

[flang][driver] Add support for `-mmlir`

The semantics of `-mmlir` are identical to `-mllvm`. The only notable
difference is that `-mmlir` options should be forwarded to MLIR rather
than LLVM.

Note

[flang][driver] Add support for `-mmlir`

The semantics of `-mmlir` are identical to `-mllvm`. The only notable
difference is that `-mmlir` options should be forwarded to MLIR rather
than LLVM.

Note that MLIR llvm::cl options are lazily constructed on demand (see
the definition of options in PassManagerOptions.cpp). This means that:
* MLIR global options are only visible when explicitly initialised and
displayed only when using `-mmlir --help`,
* Flang and LLVM global options are always visible and displayed when
using either `-mllvm -help` or `-mmlir --help`.

In other words, `-mmlir --help` is a superset of `-mllvm --help`. This is not
ideal, but we'd need to refactor all option definitions in Flang and
LLVM to improve this. I suggesting leaving this for later.

Differential Revision: https://reviews.llvm.org/D123297

show more ...

[flang][driver] Add support for generating LLVM bytecode files

Support for generating LLVM BC files is added in Flang's compiler and
frontend drivers. This requires the `BitcodeWriterPass` pass to b

[flang][driver] Add support for generating LLVM bytecode files

Support for generating LLVM BC files is added in Flang's compiler and
frontend drivers. This requires the `BitcodeWriterPass` pass to be run
on the input LLVM IR module and is implemented as a dedicated frontend
aciton. The new functionality as seen by the user (compiler driver):
```
flang-new -c -emit-llvm file.90
```
or (frontend driver):
```
flang-new -fc1 -emit-llvm-bc file.f90
```

The new behaviour is consistent with `clang` and `clang -cc1`.

Differential Revision: https://reviews.llvm.org/D123211

show more ...

[flang][driver] Add support for `-mllvm`

This option is added in both `flang-new` (the compiler driver) and
`flang-new -fc1` (the frontend driver). The semantics are consistent
with `clang` and `cla

[flang][driver] Add support for `-mllvm`

This option is added in both `flang-new` (the compiler driver) and
`flang-new -fc1` (the frontend driver). The semantics are consistent
with `clang` and `clang -cc1`.

As Flang does not run any LLVM passes when invoked with `-emit-llvm`
(i.e. `flang-new -S -emit-llvm <file>`), the tests use
`-S`/`-c`/`-emit-obj` instead. These options require an LLVM backend to
be run by the driver to generate the output (this makese `-mllvm`
relevant here).

Differential Revision: https://reviews.llvm.org/D121374

show more ...

[flang][driver] Add support for -S and implement -c/-emit-obj

This patch adds support for:
* `-S` in Flang's compiler and frontend drivers,
and implements:
* `-emit-obj` in Flang's frontend driv

[flang][driver] Add support for -S and implement -c/-emit-obj

This patch adds support for:
* `-S` in Flang's compiler and frontend drivers,
and implements:
* `-emit-obj` in Flang's frontend driver and `-c` in Flang's compiler
driver (this is consistent with Clang).
(these options were already available before, but only as placeholders).
The semantics of these options in Clang and Flang are identical.

The `EmitObjAction` frontend action is renamed as `BackendAction`. This
new name more accurately reflects the fact that this action will
primarily run the code-gen/backend pipeline in LLVM. It also makes more
sense as an action implementing both `-emit-obj` and `-S` (originally,
it was just `-emit-obj`).

`tripleName` from FirContext.cpp is deleted and, when a target triple is
required, `mlir::LLVM::LLVMDialect::getTargetTripleAttrName()` is used
instead. In practice, this means that `fir.triple` is replaced with
`llvm.target_triple`. The former was effectively ignored. The latter is
used when lowering from the LLVM dialect in MLIR to LLVM IR (i.e. it's
embedded in the generated LLVM IR module). The driver can then re-use
it when configuring the backend. With this change, the LLVM IR files
generated by e.g. `tco` will from now on contain the correct target
triple.

The code-gen.f90 test is replaced with code-gen-x86.f90 and
code-gen-aarch64.f90. With 2 seperate files we can verify that
`--target` is correctly taken into account. LIT configuration is updated
to enable e.g.:
```
! REQUIRES: aarch64-registered-target
```

Differential Revision: https://reviews.llvm.org/D120568

show more ...

[flang][driver] Add support for `-debug-dump-pft`

This patch adds support for dumping the pre-FIR tree in `flang-new
-fc1`, i.e. Flang's frontend driver. This flag is functionally identical
to `-pft

[flang][driver] Add support for `-debug-dump-pft`

This patch adds support for dumping the pre-FIR tree in `flang-new
-fc1`, i.e. Flang's frontend driver. This flag is functionally identical
to `-pft-test` in `bbc` and semantically similar to
`-fdebug-dump-parse-tree` from `flang-new -fc1`.

Differential Revision: https://reviews.llvm.org/D121198

show more ...

[flang][driver] Add support for `-emit-llvm`

This patch adds support for the `-emit-llvm` option in the frontend
driver (i.e. `flang-new -fc1`). Similarly to Clang, `flang-new -fc1
-emit-llvm file.f

[flang][driver] Add support for `-emit-llvm`

This patch adds support for the `-emit-llvm` option in the frontend
driver (i.e. `flang-new -fc1`). Similarly to Clang, `flang-new -fc1
-emit-llvm file.f` will generate a textual LLVM IR file.

Depends on D118985

Differential Revision: https://reviews.llvm.org/D119012

show more ...

[flang][driver] Add support for `-emit-mlir`

This patch adds support for generating MLIR files in Flang's frontend
driver (i.e. `flang-new -fc1`). `-emit-fir` is added as an alias for
`-emit-mlir`.

[flang][driver] Add support for `-emit-mlir`

This patch adds support for generating MLIR files in Flang's frontend
driver (i.e. `flang-new -fc1`). `-emit-fir` is added as an alias for
`-emit-mlir`. We may want to decide to split the two in the future.

A new parent class for code-gen frontend actions is introduced:
`CodeGenAction`. We will be using this class to encapsulate logic shared
between all code-generation actions, but not required otherwise. For
now, it will:
* run prescanning, parsing and semantic checks,
* lower the input to MLIR.
`EmitObjAction` is updated to inherit from this class. This means that
the behaviour of `flang-new -fc1 -emit-obj` is also updated (previously,
it would just exit immediately). This change required
`flang/test/Driver/syntax-only.f90` to be updated.

For `-emit-fir`, a specialisation of `CodeGenAction` is introduced:
`EmitMLIRAction`. The key logic for this class is implemented in
`EmitMLIRAction::ExecuteAction`.

Differential Revision: https://reviews.llvm.org/D118985

show more ...

[flang][driver] Add support for Frontend Plugins

Introducing a plugin API and a simple HelloWorld Plugin example.
This patch adds the `-load` and `-plugin` flags to frontend driver and
the code arou

[flang][driver] Add support for Frontend Plugins

Introducing a plugin API and a simple HelloWorld Plugin example.
This patch adds the `-load` and `-plugin` flags to frontend driver and
the code around using custom frontend actions from within a plugin
shared library object.

It also adds to the Driver-help test to check the help option with the
updated driver flags.

Additionally, the patch creates a plugin-example test to check the
HelloWorld plugin example runs correctly. As part of this, a new CMake
flag (`FLANG_BUILD_EXAMPLES`) is added to allow the example to be built
and for the test to run.

This Plugin API has only been tested on Linux.

Reviewed By: awarzynski

Differential Revision: https://reviews.llvm.org/D106137

show more ...

[flang][nfc] Fix variable names in `FrontendOptions` & `PreprocessorOptions`

As all member variables in `FrontendOptions` and `PreprocessorOptions`
are public, we should be naming them as `variable`

[flang][nfc] Fix variable names in `FrontendOptions` & `PreprocessorOptions`

As all member variables in `FrontendOptions` and `PreprocessorOptions`
are public, we should be naming them as `variable` rather than
`variable_` [1]. This patch fixes that.

Also, `FrontendOptions` & `PreprocessorOptions` are re-defined as a
structs rather than classes (all fields are meant to be public).

[1]
https://github.com/llvm/llvm-project/blob/main/flang/docs/C%2B%2Bstyle.md#naming

Differential Revision: https://reviews.llvm.org/D107062

show more ...

[flang] Produce proper "preprocessor output" for -E option

Rename the current -E option to "-E -Xflang -fno-reformat".

Add a new Parsing::EmitPreprocessedSource() routine to convert the
cooked char

[flang] Produce proper "preprocessor output" for -E option

Rename the current -E option to "-E -Xflang -fno-reformat".

Add a new Parsing::EmitPreprocessedSource() routine to convert the
cooked character stream output of the prescanner back to something
more closely resembling output from a traditional preprocessor;
call this new routine when -E appears.

The new -E output is suitable for use as fixed form Fortran source to
compilation by (one hopes) any Fortran compiler. If the original
top-level source file had been free form source, the output will be
suitable for use as free form source as well; otherwise there may be
diagnostics about missing spaces if they were indeed absent in the
original fixed form source.

Unless the -P option appears, #line directives are interspersed
with the output (but be advised, f18 will ignore these if presented
with them in a later compilation).

An effort has been made to preserve original alphabetic character case
and source indentation.

Add -P and -fno-reformat to the new drivers.

Tweak test options to avoid confusion with prior -E output; use
-fno-reformat where needed, but prefer to keep -E, sometimes
in concert with -P, on most, updating expected results accordingly.

Differential Revision: https://reviews.llvm.org/D106727

show more ...

[flang] Revert "PoC for Flang Driver Plugins"

This patch has not been reviewed and was commited by accident.

This reverts commit 788a5d4afe6407e647454a9832a7b4a27fba06bf.