PoC for Flang Driver Plugins

[OptTable] Rename PrintHelp to printHelp

To be consistent with other member functions and match the coding standard.

[flang][driver] Add `-fdebug-dump-all`

The new option will run the semantic checks and then dump the parse tree
and all the symbols. This is equivalent to running the driver twice,
once with `-fdebu

[flang][driver] Add `-fdebug-dump-all`

The new option will run the semantic checks and then dump the parse tree
and all the symbols. This is equivalent to running the driver twice,
once with `-fdebug-dump-parse-tree` and then with
the `-fdebug-dump-symbols` action flag.

Currently we wouldn't be able to achieve the same by simply running:
```
flang-new -fc1 -fdebug-dump-parse-tree -fdebug-dump-symbols <input-file>
```
That's because the new driver will only run one frontend action per
invocation (both of the flags used here are action flags). Diverging
from this design would lead to costly compromises and it's best avoided.

We may want to consider re-designing our debugging actions (and action
options) in the future so that there's more code re-use. For now, I'm
focusing on making sure that we support all the major cases requested by
our users.

Differential Revision: https://reviews.llvm.org/D104305

show more ...

[flang][driver] Add support for the "-init-only" option

Adding the `-init-only` option and corresponding frontend action to
generate a diagnostic.

`-init-only` vs `-test-io`:
`-init-only` ignores t

[flang][driver] Add support for the "-init-only" option

Adding the `-init-only` option and corresponding frontend action to
generate a diagnostic.

`-init-only` vs `-test-io`:
`-init-only` ignores the input (it never calls the prescanner)
`-test-io` is similar to `-init-only`, but does read and print the input
without calling the prescanner.

This patch also adds a Driver test to check this action.

Reviewed By: awarzynski, AMDChirag

Differential Revision: https://reviews.llvm.org/D102849

show more ...

[flang][driver] Add support for `-fget-definition`

This patch adds `-fget-definition` to `flang-new`. The semantics of this
option are identical in both drivers. The error message in the
"throwaway"

[flang][driver] Add support for `-fget-definition`

This patch adds `-fget-definition` to `flang-new`. The semantics of this
option are identical in both drivers. The error message in the
"throwaway" driver is updated so that it matches the one from
`flang-new` (which is auto-generated and cannot be changed easily).

Tests are updated accordingly. A dedicated test for error handling was
added: get-definition.f90 (for the sake of simplicity,
getdefinition01.f90 no longer tests for errors).

The `ParseFrontendArgs` function is updated so that it can return
errors. This change is required in order to report invalid values
following `-fget-definition`.

The actual implementation of `GetDefinitionAction::ExecuteAction()` was
extracted from f18.cpp (i.e. the bit that deals with
`-fget-definition`).

Depends on: https://reviews.llvm.org/D100556

Differential Revision: https://reviews.llvm.org/D100558

show more ...

[flang][driver] Add debug options not requiring semantic checks

This patch adds two debugging options in the new Flang driver
(flang-new):
*fdebug-unparse-no-sema
*fdebug-dump-parse-tree-no-sema

[flang][driver] Add debug options not requiring semantic checks

This patch adds two debugging options in the new Flang driver
(flang-new):
*fdebug-unparse-no-sema
*fdebug-dump-parse-tree-no-sema
Each of these options combines two options from the "throwaway" driver
(left: f18, right: flang-new):
* `-fdebug-uparse -fdebug-no-semantics` --> `-fdebug-unparse-no-sema`
* `-fdebug-dump-parse-tree -fdebug-no-semantics` -->
`-fdebug-dump-parse-tree-no-sema`

There are no plans to implement `-fdebug-no-semantics` in the new
driver. Such option would be too powerful. Also, it would only make
sense when combined with specific frontend actions (`-fdebug-unparse`
and `-fdebug-dump-parse-tree`). Instead, this patch adds 2 specialised
options listed above. Each of these is implemented through a dedicated
FrontendAction (also added).

The new frontend actions are implemented in terms of a new abstract base
action: `PrescanAndSemaAction`. This new base class was required so that
we can have finer control over what steps within the frontend are
executed:
* `PrescanAction`: run the _prescanner_
* `PrescanAndSemaAction`: run the _prescanner_ and the _parser_ (new
in this patch)
* `PrescanAndSemaAction`: run the _prescanner_, _parser_ and run the
_semantic checks_

This patch introduces `PrescanAndParseAction::BeginSourceFileAction`.
Apart from the semantic checks removed at the end, it is similar to
`PrescanAndSemaAction::BeginSourceFileAction`.

Differential Revision: https://reviews.llvm.org/D99645

show more ...

[flang][driver] Add support for `-fget-symbols-sources`

Adds support for `-fget-symbols-sources` in the new Flang driver. All
relevant tests are updated to use the new driver when
`FLANG_BUILD_NEW_D

[flang][driver] Add support for `-fget-symbols-sources`

Adds support for `-fget-symbols-sources` in the new Flang driver. All
relevant tests are updated to use the new driver when
`FLANG_BUILD_NEW_DRIVER` is set.

`RUN` lines in tests are updated so `-fsyntax-only`
comes before `-fget-symbols-sources`. That's because:
* both `-fsyntax-only` and `-fget-symbols-sources` are
action flags, and
* the new driver, flang-new, will only consider the right-most
action flag.
In other words, this change is needed so that the tests work with both
`f18` (requires both flags) and `flang-new` (only considers the last
action flag).

Differential Revision: https://reviews.llvm.org/D98191

show more ...

[flang][driver] Add `-fdebug-dump-parsing-log`

This patch adds `-fdebug-dump-parsing-log` in the new driver. This option is
semantically identical to `-fdebug-instrumented-parse` in `f18` (the
forme

[flang][driver] Add `-fdebug-dump-parsing-log`

This patch adds `-fdebug-dump-parsing-log` in the new driver. This option is
semantically identical to `-fdebug-instrumented-parse` in `f18` (the
former is added as an alias in `f18`).

As dumping the parsing log makes only sense for instrumented parses, we
set Fortran::parser::Options::instrumentedParse to `True` when
`-fdebug-dump-parsing-log` is used. This is consistent with `f18`.

To facilitate tweaking the configuration of the frontend based on the
action being requested, `setUpFrontendBasedOnAction` is introduced in
CompilerInvocation.cpp.

Differential Revision: https://reviews.llvm.org/D97457

show more ...

[flang][driver] Add debug measure-parse-tree and pre-fir-tree options

Add the following options:
* -fdebug-measure-parse-tree
* -fdebug-pre-fir-tree

Summary of changes:
- Add 2 new frontend actions

[flang][driver] Add debug measure-parse-tree and pre-fir-tree options

Add the following options:
* -fdebug-measure-parse-tree
* -fdebug-pre-fir-tree

Summary of changes:
- Add 2 new frontend actions: DebugMeasureParseTreeAction and DebugPreFIRTreeAction
- Add MeasurementVisitor to FrontendActions.h
- Make reportFatalSemanticErrors return true if there are any fatal errors
- Port most of the `-fdebug-pre-fir-tree` tests to use the new driver if built, otherwise use f18.

Differential Revision: https://reviews.llvm.org/D96884

show more ...

[flang][driver] Add debug dump options

Add the following options:
* -fdebug-dump-symbols
* -fdebug-dump-parse-tree
* -fdebug-dump-provenance

Summary of changes:
- Add 3 new frontend actions: DebugD

[flang][driver] Add debug dump options

Add the following options:
* -fdebug-dump-symbols
* -fdebug-dump-parse-tree
* -fdebug-dump-provenance

Summary of changes:
- Add 3 new frontend actions: DebugDumpSymbolsAction, DebugDumpParseTreeAction and DebugDumpProvenanceAction
- Add a unique pointer to the Semantics instance created in PrescanAndSemaAction
- Move fatal semantic error reporting to its own method, FrontendActions#reportFatalSemanticErrors
- Port most tests using `-fdebug-dump-symbols` and `-fdebug-dump-parse-tree` to the new driver if built, otherwise default to f18

Differential Revision: https://reviews.llvm.org/D96716

show more ...

[flang][driver] Add options for unparsing

This patch adds the following compiler frontend driver options:
* -fdebug-unparse (f18 spelling: -funparse)
* -fdebug-unparse-with-symbols (f18 spelling

[flang][driver] Add options for unparsing

This patch adds the following compiler frontend driver options:
* -fdebug-unparse (f18 spelling: -funparse)
* -fdebug-unparse-with-symbols (f18 spelling: -funparse-with-symbols)
The new driver will only accept the new spelling. `f18` will accept both
the original and the new spelling.

A new base class for frontend actions is added: `PrescanAndSemaAction`.
This is added to reduce code duplication that otherwise these new
options would lead to. Implementation from
* `ParseSyntaxOnlyAction::ExecutionAction`
is moved to:
* `PrescanAndSemaAction::BeginSourceFileAction`
This implementation is now shared between:
* PrescanAndSemaAction
* ParseSyntaxOnlyAction
* DebugUnparseAction
* DebugUnparseWithSymbolsAction

All tests that don't require other yet unimplemented options are
updated. This way `flang-new -fc1` is used instead of `f18` when
`FLANG_BUILD_NEW_DRIVER` is set to `On`. In order to facilitate this,
`%flang_fc1` is added in the LIT configuration (lit.cfg.py).

`asFortran` from f18.cpp is duplicated as `getBasicAsFortran` in
FrontendOptions.cpp. At this stage it's hard to find a good place to
share this method. I suggest that we revisit this once a switch from
`f18` to `flang-new` is complete.

Differential Revision: https://reviews.llvm.org/D96483

show more ...

[flang][driver] Move isFixedFormSuffix and isFreeFormSuffix to flangFrontend

isFixedFormSuffix and isFreeFormSuffix should be defined in
flangFrontend rather than flangFrontendTool library. That's f

[flang][driver] Move isFixedFormSuffix and isFreeFormSuffix to flangFrontend

isFixedFormSuffix and isFreeFormSuffix should be defined in
flangFrontend rather than flangFrontendTool library. That's for 2
reasons:
* these methods are used in flangFrontend rather than flangFrontendTool
* flangFrontendTool depends on flangFrontend

As mentioned in the post-commit review for D94228, without this change
shared library builds fail.

Differential Revision: https://reviews.llvm.org/D94968

show more ...

[flang][driver] Add support for fixed form detection

Currently the new flang driver always runs in free form mode. This patch
adds support for fixed form mode detection based on the file extensions.

[flang][driver] Add support for fixed form detection

Currently the new flang driver always runs in free form mode. This patch
adds support for fixed form mode detection based on the file extensions.

Like `f18`, `flang-new` will treat files ending with ".f", ".F" and
".ff" as fixed form. Additionally, ".for", ".FOR", ".fpp" and ".FPP"
file extensions are recognised as fixed form files. This is consistent
with gfortran [1]. In summary, files with the following extensions are
treated as fixed-form:
* ".f", ".F", ".ff", ".for", ".FOR", ".fpp", ".FPP"

For consistency with flang/test/lit.cfg.py and f18, this patch also adds
support for the following file extensions:
* ".ff", ".FOR", ".for", ".ff90", ".fpp", ".FPP"
This is added in flang/lib/Frontend/FrontendOptions.cpp. Additionally,
the following extensions are included:
* ".f03", ".F03", ".f08", ".F08"
This is for compatibility with gfortran [1] and other popular Fortran
compilers [2].

NOTE: internally Flang will only differentiate between fixed and free
form files. Currently Flang does not support switching between language
standards, so in this regard file extensions are irrelevant. More
specifically, both `file.f03` and `file.f18` are represented with
`Language::Fortran` (as opposed to e.g. `Language::Fortran03`).

Summary of changes:
- Set Fortran::parser::Options::sFixedForm according to the file type
- Add isFixedFormSuffix and isFreeFormSuffix helper functions to
FrontendTool/Utils.h
- Change FrontendOptions::GetInputKindForExtension to support the missing
file extensions that f18 supports and some additional ones
- FrontendActionTest.cpp is updated to make sure that the test input is
treated as free-form

[1] https://gcc.gnu.org/onlinedocs/gfortran/GNU-Fortran-and-GCC.html
[2] https://github.com/llvm/llvm-project/blob/master/flang/docs/OptionComparison.md#notes

Differential Revision: https://reviews.llvm.org/D94228

show more ...

[flang][driver] Add support for `-c` and `-emit-obj`

This patch adds a frontend action for emitting object files. While Flang
does not support code-generation, this action remains a placeholder.
Thi

[flang][driver] Add support for `-c` and `-emit-obj`

This patch adds a frontend action for emitting object files. While Flang
does not support code-generation, this action remains a placeholder.
This patch simply provides glue-code to connect the compiler driver
with the appropriate frontend action.

The new action is triggered with the `-c` compiler driver flag, i.e.
`flang-new -c`. This is then translated to `flang-new -fc1 -emit-obj`,
so `-emit-obj` has to be marked as supported as well.

As code-generation is not available yet, `flang-new -c` results in a
driver error:
```
error: code-generation is not available yet
```
Hopefully this will help communicating the level of available
functionality within Flang.

The definition of `emit-obj` is updated so that it can be shared between
Clang and Flang. As the original definition was enclosed within a
Clang-specific TableGen `let` statement, it is extracted into a new `let`
statement. That felt like the cleanest option.

I also commented out `-triple` in Flang::ConstructJob and updated some
comments there. This is similar to https://reviews.llvm.org/D93027. I
wanted to make sure that it's clear that we can't support `-triple`
until we have code-generation. However, once code-generation is
available we _will need_ `-triple`.

As this patch adds `-emit-obj`, the emit-obj.f90 becomes irrelevant and
is deleted. Instead, phases.f90 is added to demonstrate that users can
control compilation phases (indeed, `-c` is a phase control flag).

Reviewed By: SouraVX, clementval

Differential Revision: https://reviews.llvm.org/D93301

show more ...

[flang][driver] Add support for `-fsyntax-only`

The behaviour triggered with this flag is consistent with `-fparse-only`
in `flang` (i.e. the throwaway driver). This new spelling is consistent
with

[flang][driver] Add support for `-fsyntax-only`

The behaviour triggered with this flag is consistent with `-fparse-only`
in `flang` (i.e. the throwaway driver). This new spelling is consistent
with Clang and gfortran, and was proposed and agreed on for the new
driver in [1].

This patch also adds some minimal logic to communicate whether the
semantic checks have failed or not. When semantic checks fail, a
frontend driver error is generated. The return code from the frontend
driver is then determined by checking the driver diagnostics - the
presence of driver errors means that the compilation has failed. This
logic is consistent with `clang -cc1`.

[1] http://lists.llvm.org/pipermail/flang-dev/2020-November/000588.html

Differential Revision: https://reviews.llvm.org/D92854

show more ...

[Flang][Driver] Add PrintPreprocessedInput FrontendAction (`flang-new -E`)

This patch implements the first frontend action for the Flang parser (i.e.
Fortran::parser). This action runs the preproces

[Flang][Driver] Add PrintPreprocessedInput FrontendAction (`flang-new -E`)

This patch implements the first frontend action for the Flang parser (i.e.
Fortran::parser). This action runs the preprocessor and is invoked with the
`-E` flag. (i.e. `flang-new -E <input-file>). The generated output is printed
to either stdout or the output file (specified with `-` or `-o <output-file>`).

Note that currently there is no mechanism to map options for the
frontend driver (i.e. Fortran::frontend::FrontendOptions) to options for
the parser (i.e. Fortran::parser::Options). Instead,
Frotran::parser::options are hard-coded to:

```
std::vector<std::string> searchDirectories{"."s};
searchDirectories = searchDirectories;
isFixedForm = false;
_encoding(Fortran::parser::Encoding::UTF_8);
```

These default settings are compatible with the current Flang driver. Further
work is required in order for CompilerInvocation to read and map
clang::driver::options to Fortran::parser::options.

Co-authored-by: Andrzej Warzynski <andrzej.warzynski@arm.com>

Differential Revision: https://reviews.llvm.org/D88381

show more ...

[flang][driver] Rename the accessors/mutators (NFC)

As per point 3 in [1]:

```
Accessor member functions are named with the non-public data member's
name, less the trailing underscore. Mutator mem

[flang][driver] Rename the accessors/mutators (NFC)

As per point 3 in [1]:

```
Accessor member functions are named with the non-public data member's
name, less the trailing underscore. Mutator member functions are named
set_...
```

Originally we just followed the LLVM's style, which is incompatible with
Flang. This patch renames the accessors and mutators accordingly.

`getDiagnostics` and `GetDiagnostics` are replaced with one accessor:
`diagnostics`. `SetDiagnostics` was neither implemented nor used, so
it's deleted.

[1] https://github.com/llvm/llvm-project/blob/master/flang/docs/C++style.md#naming

Differential Revision: https://reviews.llvm.org/D90300

show more ...

[Flang][Driver] Add infrastructure for basic frontend actions and file I/O

This patch introduces the dependencies required to read and manage input files
provided by the command line option. It also

[Flang][Driver] Add infrastructure for basic frontend actions and file I/O

This patch introduces the dependencies required to read and manage input files
provided by the command line option. It also adds the infrastructure to create
and write to output files. The output is sent to either stdout or a file
(specified with the `-o` flag).

Separately, in order to be able to test the code for file I/O, it adds
infrastructure to create frontend actions. As a basic testable example, it adds
the `InputOutputTest` FrontendAction. The sole purpose of this action is to
read a file from the command line and print it either to stdout or the output
file. This action is run by using the `-test-io` flag also introduced in this
patch (available for `flang-new` and `flang-new -fc1`). With this patch:
```
flang-new -test-io input-file.f90
```
will read input-file.f90 and print it in the output file.

The `InputOutputTest` frontend action has been introduced primarily to
facilitate testing. It is hidden from users (i.e. it's only displayed with
`--help-hidden`). Currently Clang doesn’t have an equivalent action.

`-test-io` is used to trigger the InputOutputTest action in the Flang frontend
driver. This patch makes sure that “flang-new” forwards it to “flang-new -fc1"
by creating a preprocessor job. However, in Flang.cpp, `-test-io` is passed to
“flang-new -fc1” without `-E`. This way we make sure that the preprocessor is
_not_ run in the frontend driver. This is the desired behaviour: `-test-io`
should only read the input file and print it to the output stream.

co-authored-by: Andrzej Warzynski <andrzej.warzynski@arm.com>

Differential Revision: https://reviews.llvm.org/D87989

show more ...

[flang][driver] Fix options flag in the frontend driver

In the frontend driver we should be using FC1Option (frontend driver
options) instead of FlangOption (flang driver options).

Differential Rev

[flang][driver] Fix options flag in the frontend driver

In the frontend driver we should be using FC1Option (frontend driver
options) instead of FlangOption (flang driver options).

Differential Revision: https://reviews.llvm.org/D88108

show more ...

[flang][driver] Add the new flang compiler and frontend drivers

Summary:

This is the first patch implementing the new Flang driver as outlined in [1],
[2] & [3]. It creates Flang driver (`flang-new

[flang][driver] Add the new flang compiler and frontend drivers

Summary:

This is the first patch implementing the new Flang driver as outlined in [1],
[2] & [3]. It creates Flang driver (`flang-new`) and Flang frontend driver
(`flang-new -fc1`). These will be renamed as `flang` and `flang -fc1` once the
current Flang throwaway driver, `flang`, can be replaced with `flang-new`.

Currently only 2 options are supported: `-help` and `--version`.

`flang-new` is implemented in terms of libclangDriver, defaulting the driver
mode to `FlangMode` (added to libclangDriver in [4]). This ensures that the
driver runs in Flang mode regardless of the name of the binary inferred from
argv[0].

The design of the new Flang compiler and frontend drivers is inspired by it
counterparts in Clang [3]. Currently, the new Flang compiler and frontend
drivers re-use Clang libraries: clangBasic, clangDriver and clangFrontend.

To identify Flang options, this patch adds FlangOption/FC1Option enums.
Driver::printHelp is updated so that `flang-new` prints only Flang options.
The new Flang driver is disabled by default. To enable it, set
`-DBUILD_FLANG_NEW_DRIVER=ON` when configuring CMake and add clang to
`LLVM_ENABLE_PROJECTS` (e.g. -DLLVM_ENABLE_PROJECTS=“clang;flang;mlir”).

[1] “RFC: new Flang driver - next steps”
http://lists.llvm.org/pipermail/flang-dev/2020-July/000470.html
[2] “RFC: Adding a fortran mode to the clang driver for flang”
http://lists.llvm.org/pipermail/cfe-dev/2019-June/062669.html
[3] “RFC: refactoring libclangDriver/libclangFrontend to share with Flang”
http://lists.llvm.org/pipermail/cfe-dev/2020-July/066393.html
[4] https://reviews.llvm.org/rG6bf55804924d5a1d902925ad080b1a2b57c5c75c

co-authored-by: Andrzej Warzynski <andrzej.warzynski@arm.com>

Reviewed By: richard.barton.arm, sameeranjoshi

Differential Revision: https://reviews.llvm.org/D86089

show more ...