-ftime-report: Move FrontendTimer closer to TimeTraceScope

... to improve consistency and make "Clang time report" cover
`FrontendAction::BeginSourceFile` and `FrontendAction::EndSourceFile`.

-ftime-report: reorganize timers

The code generation time is unclear in the -ftime-report output:

* The two clang timers "Code Generation Time" and "LLVM IR Generation
Time" are in the default gr

-ftime-report: reorganize timers

The code generation time is unclear in the -ftime-report output:

* The two clang timers "Code Generation Time" and "LLVM IR Generation
Time" are in the default group "Miscellaneous Ungrouped Timers".
* There is also a "Clang front-end time" group, which actually includes
code generation time.

```
===-------------------------------------------------------------------------===
Miscellaneous Ungrouped Timers
===-------------------------------------------------------------------------===

---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name ---
0.0611 ( 1.7%) 0.0099 ( 4.4%) 0.0710 ( 1.9%) 0.0713 ( 1.9%) LLVM IR Generation Time
3.5140 ( 98.3%) 0.2165 ( 95.6%) 3.7306 ( 98.1%) 3.7342 ( 98.1%) Code Generation Time
3.5751 (100.0%) 0.2265 (100.0%) 3.8016 (100.0%) 3.8055 (100.0%) Total
...
===-------------------------------------------------------------------------===
Clang front-end time report
===-------------------------------------------------------------------------===
Total Execution Time: 3.9108 seconds (3.9146 wall clock)

---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name ---
3.6802 (100.0%) 0.2306 (100.0%) 3.9108 (100.0%) 3.9146 (100.0%) Clang front-end timer
3.6802 (100.0%) 0.2306 (100.0%) 3.9108 (100.0%) 3.9146 (100.0%) Total
```

This patch

* renames "Clang front-end time report" (FrontendAction time) to "Clang
time report",
* renames "Clang front-end" to "Front end",
* moves "LLVM IR Generation" into the group,
* replaces "Code Generation time" with "Optimizer" (middle end) and
"Machine code generation" (back end).

```
% clang -c sqlite3.i -w -ftime-report -mllvm -sort-timers=0
...
===-------------------------------------------------------------------------===
Clang time report
===-------------------------------------------------------------------------===
Total Execution Time: 1.5922 seconds (1.5972 wall clock)

---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name ---
0.5107 ( 35.9%) 0.0105 ( 6.2%) 0.5211 ( 32.7%) 0.5222 ( 32.7%) Front end
0.2464 ( 17.3%) 0.0340 ( 20.0%) 0.2804 ( 17.6%) 0.2814 ( 17.6%) LLVM IR generation
0.6240 ( 43.9%) 0.1235 ( 72.7%) 0.7475 ( 47.0%) 0.7503 ( 47.0%) Machine code generation
0.0413 ( 2.9%) 0.0018 ( 1.0%) 0.0431 ( 2.7%) 0.0433 ( 2.7%) Optimizer
1.4224 (100.0%) 0.1698 (100.0%) 1.5922 (100.0%) 1.5972 (100.0%) Total
```

Pull Request: https://github.com/llvm/llvm-project/pull/122225

show more ...

Reapply "[NFC] Explicitly pass a VFS when creating DiagnosticsEngine (#115852)"

This reverts commit a1153cd6fedd4c906a9840987934ca4712e34cb2 with fixes
to lldb breakages.

Fixes https://github.com/l

Reapply "[NFC] Explicitly pass a VFS when creating DiagnosticsEngine (#115852)"

This reverts commit a1153cd6fedd4c906a9840987934ca4712e34cb2 with fixes
to lldb breakages.

Fixes https://github.com/llvm/llvm-project/issues/117145.

show more ...

Revert "[NFC] Explicitly pass a VFS when creating DiagnosticsEngine (#115852)"

Reverted for causing:
https://github.com/llvm/llvm-project/issues/117145

This reverts commit bdd10d9d249bd1c2a45e3de56

Revert "[NFC] Explicitly pass a VFS when creating DiagnosticsEngine (#115852)"

Reverted for causing:
https://github.com/llvm/llvm-project/issues/117145

This reverts commit bdd10d9d249bd1c2a45e3de56a5accd97e953458.

show more ...

[NFC] Explicitly pass a VFS when creating DiagnosticsEngine (#115852)

Starting with 41e3919ded78d8870f7c95e9181c7f7e29aa3cc4 DiagnosticsEngine
creation might perform IO. It was implicitly defaultin

[NFC] Explicitly pass a VFS when creating DiagnosticsEngine (#115852)

Starting with 41e3919ded78d8870f7c95e9181c7f7e29aa3cc4 DiagnosticsEngine
creation might perform IO. It was implicitly defaulting to
getRealFileSystem. This patch makes it explicit by pushing the decision
making to callers.

It uses ambient VFS if one is available, and keeps using
`getRealFileSystem` if there aren't any VFS.

show more ...

Reapply "[clang] Introduce diagnostics suppression mappings (#112517)"

This reverts commit 5f140ba54794fe6ca379362b133eb27780e363d7.

Revert "[clang] Introduce diagnostics suppression mappings (#112517)"

This reverts commit 12e3ed8de8c6063b15916b3faf67c8c9cd17df1f.
This reverts commit 41e3919ded78d8870f7c95e9181c7f7e29aa3cc4.

The

Revert "[clang] Introduce diagnostics suppression mappings (#112517)"

This reverts commit 12e3ed8de8c6063b15916b3faf67c8c9cd17df1f.
This reverts commit 41e3919ded78d8870f7c95e9181c7f7e29aa3cc4.

There are some buildbot breakages in
https://lab.llvm.org/buildbot/#/builders/18/builds/6832.

show more ...

[clang] Introduce diagnostics suppression mappings (#112517)

This implements

https://discourse.llvm.org/t/rfc-add-support-for-controlling-diagnostics-severities-at-file-level-granularity-through-

[clang] Introduce diagnostics suppression mappings (#112517)

This implements

https://discourse.llvm.org/t/rfc-add-support-for-controlling-diagnostics-severities-at-file-level-granularity-through-command-line/81292.

Users now can suppress warnings for certain headers by providing a
mapping with globs, a sample file looks like:
```
[unused]
src:*
src:*clang/*=emit
```

This will suppress warnings from `-Wunused` group in all files that
aren't under `clang/` directory. This mapping file can be passed to
clang via `--warning-suppression-mappings=foo.txt`.

At a high level, mapping file is stored in DiagnosticOptions and then
processed with rest of the warning flags when creating a
DiagnosticsEngine. This is a functor that uses SpecialCaseLists
underneath to match against globs coming from the mappings file.

This implies processing warning options now performs IO, relevant
interfaces are updated to take in a VFS, falling back to RealFileSystem
when one is not available.

show more ...

[clang] Make deprecations of some `FileManager` APIs formal (#110014)

Some `FileManager` APIs still return `{File,Directory}Entry` instead of
the preferred `{File,Directory}EntryRef`. These are doc

[clang] Make deprecations of some `FileManager` APIs formal (#110014)

Some `FileManager` APIs still return `{File,Directory}Entry` instead of
the preferred `{File,Directory}EntryRef`. These are documented to be
deprecated, but don't have the attribute that warns on their usage. This
PR marks them as such with `LLVM_DEPRECATED()` and replaces their usage
with the recommended counterparts. NFCI.

show more ...

[clang] Tidy uses of raw_string_ostream (NFC)

As specified in the docs,
1) raw_string_ostream is always unbuffered and
2) the underlying buffer may be used directly

( 65b13610a5226b84889b923bae884b

[clang] Tidy uses of raw_string_ostream (NFC)

As specified in the docs,
1) raw_string_ostream is always unbuffered and
2) the underlying buffer may be used directly

( 65b13610a5226b84889b923bae884ba395ad084d for further reference )

* Don't call raw_string_ostream::flush(), which is essentially a no-op.
* Avoid unneeded calls to raw_string_ostream::str(), to avoid excess indirection.

show more ...

[clang][deps] Print tracing VFS data (#108056)

Clang's `-cc1 -print-stats` shows lots of useful internal data including
basic `FileManager` stats. Since this layer caches some results, it is
uncle

[clang][deps] Print tracing VFS data (#108056)

Clang's `-cc1 -print-stats` shows lots of useful internal data including
basic `FileManager` stats. Since this layer caches some results, it is
unclear how that information translates to actual filesystem accesses.
This PR uses `llvm::vfs::TracingFileSystem` to provide that missing
information.

Similar mechanism is implemented for `clang-scan-deps`'s verbose mode
(`-v`). IO contention proved to be a real bottleneck a couple of times
already and this new feature should make those easier to detect in the
future. The tracing VFS is inserted below the caching FS and above the
real FS.

show more ...

[clang] Stop adjusting the module cache path (#102540)

This patch stops adjustments of the module cache path beyond what is
done in `ParseHeaderSearchArgs` (making it absolute and removing dots).

[clang] Stop adjusting the module cache path (#102540)

This patch stops adjustments of the module cache path beyond what is
done in `ParseHeaderSearchArgs` (making it absolute and removing dots).
This enables more efficient implementation of the caching VFS in
https://github.com/llvm/llvm-project/pull/88800.

show more ...

[clang][NFC] Further improvements to const-correctness

[clang-scan-deps] Fix contention when updating `TrackingStatistic`s in hot code paths in `FileManager`. (#88427)

`FileManager::getDirectoryRef()` and `FileManager::getFileRef()` are hot code paths i

[clang-scan-deps] Fix contention when updating `TrackingStatistic`s in hot code paths in `FileManager`. (#88427)

`FileManager::getDirectoryRef()` and `FileManager::getFileRef()` are hot code paths in `clang-scan-deps`. These functions are updating on every call a few atomics related to printing statistics, which causes contention on high core count machines.

![Screenshot 2024-04-10
214123](https://github.com/llvm/llvm-project/assets/37383324/5756b1bc-cab5-4612-8769-ee7e03a66479)

![Screenshot 2024-04-10
214246](https://github.com/llvm/llvm-project/assets/37383324/3d560e89-61c7-4fb9-9330-f9e660e8fc8b)

![Screenshot 2024-04-10
214315](https://github.com/llvm/llvm-project/assets/37383324/006341fc-49d4-4720-a348-7af435c21b17)

After this patch we make the variables local to the `FileManager`.

In our test case, this saves about 49 sec over 1 min 47 sec of `clang-scan-deps` run time (1 min 47 sec before, 58 sec after). These figures are after applying my suggestion in https://github.com/llvm/llvm-project/pull/88152#issuecomment-2049803229, that is:
```
static bool shouldCacheStatFailures(StringRef Filename) {
return true;
}
```
Without the above, there's just too much OS noise from the high volume of `status()` calls with regular non-modules C++ code. Tested on Windows with clang-cl.

show more ...

[clang] Move state out of `PreprocessorOptions` (2/n) (#87099)

An instance of `PreprocessorOptions` is part of `CompilerInvocation`
which is supposed to be a value type. The `FailedModules` member

[clang] Move state out of `PreprocessorOptions` (2/n) (#87099)

An instance of `PreprocessorOptions` is part of `CompilerInvocation`
which is supposed to be a value type. The `FailedModules` member is
problematic, since it's essentially a shared state used by multiple
`CompilerInstance` objects, and not really a preprocessor option. Let's
move it into `CompilerInstance` instead.

show more ...

[clang][modules] Avoid calling expensive `SourceManager::translateFile()` (#86216)

The `ASTWriter` algorithm for computing affecting module maps uses
`SourceManager::translateFile()` to get a `File

[clang][modules] Avoid calling expensive `SourceManager::translateFile()` (#86216)

The `ASTWriter` algorithm for computing affecting module maps uses
`SourceManager::translateFile()` to get a `FileID` from a `FileEntry`.
This is slow (O(n)) since the function performs a linear walk over
`SLocEntries` until it finds one with a matching `FileEntry`.

This patch removes this use of `SourceManager::translateFile()` by
tracking `FileID` instead of `FileEntry` in couple of places in
`ModuleMap`, giving `ASTWriter` the desired `FileID` directly. There are
no changes required for clients that still want a `FileEntry` from
`ModuleMap`: the existing APIs internally use `SourceManager` to perform
the reverse `FileID` to `FileEntry` conversion in O(1).

show more ...

[AIX][TOC] Add -mtocdata/-mno-tocdata options on AIX (#67999)

This patch enables support that the XL compiler had for AIX under
-qdatalocal/-qdataimported.

[clang] Diagnose config_macros before building modules (#83641)

Before this patch, if a module fails to build because of a missing
config_macro, the user will never see the config macro warning. Th

[clang] Diagnose config_macros before building modules (#83641)

Before this patch, if a module fails to build because of a missing
config_macro, the user will never see the config macro warning. This
patch diagnoses this before building, and each subsequent time a module
is imported.

rdar://123921931

show more ...

[clang] NFC: Extract `CompilerInstance` function

[PGO] Reland PGO's Counter Reset and File Dumping APIs #76471 (#78285)

https://github.com/llvm/llvm-project/pull/76471 caused buildbot failures
on Windows. For more details, see
https://github.com

[PGO] Reland PGO's Counter Reset and File Dumping APIs #76471 (#78285)

https://github.com/llvm/llvm-project/pull/76471 caused buildbot failures
on Windows. For more details, see
https://github.com/llvm/llvm-project/issues/77546.

This PR revises the test and relands
https://github.com/llvm/llvm-project/pull/76471.

show more ...

[Frontend] Use SmallString::operator std::string (NFC)

[clang] Use SmallString::operator std::string() (NFC)

Revert "[PGO] Exposing PGO's Counter Reset and File Dumping APIs (#76471)"

Issue #77546

This reverts commit 07c9189fcc063bdf6219d2733843c89cde3991e1.

[PGO] Exposing PGO's Counter Reset and File Dumping APIs (#76471)

This PR exposes four PGO functions

- `__llvm_profile_set_filename`
- `__llvm_profile_reset_counters`,
- `__llvm_profile_dump`

[PGO] Exposing PGO's Counter Reset and File Dumping APIs (#76471)

This PR exposes four PGO functions

- `__llvm_profile_set_filename`
- `__llvm_profile_reset_counters`,
- `__llvm_profile_dump`
- `__llvm_orderfile_dump`

to user programs through the new header `instr_prof_interface.h` under
`compiler-rt/include/profile`. This way, the user can include the header
`profile/instr_prof_interface.h` to introduce these four names to their
programs.

Additionally, this PR defines macro `__LLVM_INSTR_PROFILE_GENERATE` when
the program is compiled with profile generation, and defines macro
`__LLVM_INSTR_PROFILE_USE` when the program is compiled with profile
use. `__LLVM_INSTR_PROFILE_GENERATE` together with
`instr_prof_interface.h` define the PGO functions only when the program
is compiled with profile generation. When profile generation is off,
these PGO functions are defined away and leave no trace in the user's
program.

Background:
https://discourse.llvm.org/t/pgo-are-the-llvm-profile-functions-stable-c-apis-across-llvm-releases/75832

show more ...

[clang] NFC: Remove `OptionalFileEntryRefDegradesToFileEntryPtr` (#74899)