Reapply "[NFC] Explicitly pass a VFS when creating DiagnosticsEngine (#115852)"

This reverts commit a1153cd6fedd4c906a9840987934ca4712e34cb2 with fixes
to lldb breakages.

Fixes https://github.com/l

Reapply "[NFC] Explicitly pass a VFS when creating DiagnosticsEngine (#115852)"

This reverts commit a1153cd6fedd4c906a9840987934ca4712e34cb2 with fixes
to lldb breakages.

Fixes https://github.com/llvm/llvm-project/issues/117145.

show more ...

Revert "[NFC] Explicitly pass a VFS when creating DiagnosticsEngine (#115852)"

Reverted for causing:
https://github.com/llvm/llvm-project/issues/117145

This reverts commit bdd10d9d249bd1c2a45e3de56

Revert "[NFC] Explicitly pass a VFS when creating DiagnosticsEngine (#115852)"

Reverted for causing:
https://github.com/llvm/llvm-project/issues/117145

This reverts commit bdd10d9d249bd1c2a45e3de56a5accd97e953458.

show more ...

[NFC] Explicitly pass a VFS when creating DiagnosticsEngine (#115852)

Starting with 41e3919ded78d8870f7c95e9181c7f7e29aa3cc4 DiagnosticsEngine
creation might perform IO. It was implicitly defaultin

[NFC] Explicitly pass a VFS when creating DiagnosticsEngine (#115852)

Starting with 41e3919ded78d8870f7c95e9181c7f7e29aa3cc4 DiagnosticsEngine
creation might perform IO. It was implicitly defaulting to
getRealFileSystem. This patch makes it explicit by pushing the decision
making to callers.

It uses ambient VFS if one is available, and keeps using
`getRealFileSystem` if there aren't any VFS.

show more ...

[clang] Use {} instead of std::nullopt to initialize empty ArrayRef (#109399)

Follow up to #109133.

[Frontend] Teach LoadFromASTFile to take FileName by StringRef (NFC) (#109583)

Without this patch, several callers of LoadFromASTFile construct an
instance of std::string to be passed as FileName,

[Frontend] Teach LoadFromASTFile to take FileName by StringRef (NFC) (#109583)

Without this patch, several callers of LoadFromASTFile construct an
instance of std::string to be passed as FileName, only to be converted
back to StringRef when LoadFromASTFile calls ReadAST.

This patch changes the type of FileName to StringRef and updates the
callers.

show more ...

[clang] Remove unused argument. NFC. (#73594)

[clang] NFCI: Use `FileEntryRef` in `ASTReader::GetHeaderFileInfo()`

This is the `ASTReader` counterpart to PR #67383.

[clang] Fix leak in LoadFromCommandLineWorkingDirectory unit test

Change `ASTUnit::LoadFromCommandLine` to return a `std::unique_ptr` instead of a +1 pointer, fixing a leak in the unit test `LoadFro

[clang] Fix leak in LoadFromCommandLineWorkingDirectory unit test

Change `ASTUnit::LoadFromCommandLine` to return a `std::unique_ptr` instead of a +1 pointer, fixing a leak in the unit test `LoadFromCommandLineWorkingDirectory`.

Reviewed By: bnbarham, benlangmuir

Differential Revision: https://reviews.llvm.org/D154257

show more ...

[clang] Fix ASTUnit working directory handling

Fix a couple of issues with the handling of the current working directory in ASTUnit:

- Use `createPhysicalFileSystem` instead of `getRealFileSystem`

[clang] Fix ASTUnit working directory handling

Fix a couple of issues with the handling of the current working directory in ASTUnit:

- Use `createPhysicalFileSystem` instead of `getRealFileSystem` to avoid affecting the process' current working directory, and set it at the top of `ASTUnit::LoadFromCommandLine` such that the driver used for argument parsing and the ASTUnit share the same VFS. This ensures that '-working-directory' correctly sets the VFS working directory in addition to the FileManager working directory.
- Ensure we preserve the FileSystemOptions set on the FileManager when re-creating it (as `ASTUnit::Reparse` will clear the currently set FileManager).

rdar://110697657

Reviewed By: bnbarham, benlangmuir

Differential Revision: https://reviews.llvm.org/D154134

show more ...

[C++20] [Modules] Don't ignore -fmodule-file when we compile pcm files

Close https://github.com/llvm/llvm-project/issues/62843.

Previously when we compile .pcm files into .o files, the
`-fmodule-fi

[C++20] [Modules] Don't ignore -fmodule-file when we compile pcm files

Close https://github.com/llvm/llvm-project/issues/62843.

Previously when we compile .pcm files into .o files, the
`-fmodule-file=<module-name>=<module-path>` option is ignored. This is
conflicted with our consensus in
https://github.com/llvm/llvm-project/issues/62707.

show more ...

[libclang] Add index option to store preambles in memory

This commit allows libclang API users to opt into storing PCH in memory
instead of temporary files. The option can be set only during CXIndex

[libclang] Add index option to store preambles in memory

This commit allows libclang API users to opt into storing PCH in memory
instead of temporary files. The option can be set only during CXIndex
construction to avoid multithreading issues and confusion or bugs if
some preambles are stored in temporary files and others - in memory.

The added API works as expected in KDevelop:
https://invent.kde.org/kdevelop/kdevelop/-/merge_requests/283

Differential Revision: https://reviews.llvm.org/D145974

show more ...

[libclang] Add API to override preamble storage path

TempPCHFile::create() calls llvm::sys::fs::createTemporaryFile() to
create a file named preamble-*.pch in a system temporary directory. This
comm

[libclang] Add API to override preamble storage path

TempPCHFile::create() calls llvm::sys::fs::createTemporaryFile() to
create a file named preamble-*.pch in a system temporary directory. This
commit allows overriding the directory where these often many and large
preamble-*.pch files are stored.

The referenced bug report requests the ability to override the temporary
directory path used by libclang. However, overriding the return value of
llvm::sys::path::system_temp_directory() was rejected during code review
as improper and because it would negatively affect multithreading
performance. Finding all places where libclang uses the temporary
directory is very difficult. Therefore this commit is limited to
override libclang's single known use of the temporary directory.

This commit allows to override the preamble storage path only during
CXIndex construction to avoid multithreading issues and ensure that all
preambles are stored in the same directory. For the same multithreading
and consistency reasons, this commit deprecates
clang_CXIndex_setGlobalOptions() and
clang_CXIndex_setInvocationEmissionPathOption() in favor of specifying
these options during CXIndex construction.

Adding a new CXIndex constructor function each time a new initialization
argument is needed leads to either a large number of function parameters
unneeded by most libclang users or to an exponential number of overloads
that support different usage requirements. Therefore this commit
introduces a new extensible struct CXIndexOptions and a general function
clang_createIndexWithOptions().

A libclang user passes a desired preamble storage path to
clang_createIndexWithOptions(), which stores it in
CIndexer::PreambleStoragePath. Whenever
clang_parseTranslationUnit_Impl() is called, it passes
CIndexer::PreambleStoragePath to ASTUnit::LoadFromCommandLine(), which
stores this argument in ASTUnit::PreambleStoragePath. Whenever
ASTUnit::getMainBufferWithPrecompiledPreamble() is called, it passes
ASTUnit::PreambleStoragePath to PrecompiledPreamble::Build().
PrecompiledPreamble::Build() forwards the corresponding StoragePath
argument to TempPCHFile::create(). If StoragePath is not empty,
TempPCHFile::create() stores the preamble-*.pch file in the directory at
the specified path rather than in the system temporary directory.

The analysis below proves that this passing around of the
PreambleStoragePath string is sufficient to guarantee that the libclang
user override is used in TempPCHFile::create(). The analysis ignores API
uses in test code.

TempPCHFile::create() is called only in PrecompiledPreamble::Build().
PrecompiledPreamble::Build() is called only in two places: one in
clangd, which is not used by libclang, and one in
ASTUnit::getMainBufferWithPrecompiledPreamble().
ASTUnit::getMainBufferWithPrecompiledPreamble() is called in 3 places:

ASTUnit::LoadFromCompilerInvocation() [analyzed below].
ASTUnit::Reparse(), which in turn is called only from
clang_reparseTranslationUnit_Impl(), which in turn is called only from
clang_reparseTranslationUnit(). clang_reparseTranslationUnit() is never
called in LLVM code, but is part of public libclang API. This function's
documentation requires its translation unit argument to have been built
with clang_createTranslationUnitFromSourceFile().
clang_createTranslationUnitFromSourceFile() delegates its work to
clang_parseTranslationUnit(), which delegates to
clang_parseTranslationUnit2(), which delegates to
clang_parseTranslationUnit2FullArgv(), which delegates to
clang_parseTranslationUnit_Impl(), which passes
CIndexer::PreambleStoragePath to the ASTUnit it creates.

ASTUnit::CodeComplete() passes AllowRebuild = false to
ASTUnit::getMainBufferWithPrecompiledPreamble(), which makes it return
nullptr before calling PrecompiledPreamble::Build().

Both ASTUnit::LoadFromCompilerInvocation() overloads (one of which
delegates its work to another) call
ASTUnit::getMainBufferWithPrecompiledPreamble() only if their argument
PrecompilePreambleAfterNParses > 0. LoadFromCompilerInvocation() is
called in:

ASTBuilderAction::runInvocation() keeps the default parameter value
of PrecompilePreambleAfterNParses = 0, meaning that the preamble file is
never created from here.

ASTUnit::LoadFromCommandLine().
ASTUnit::LoadFromCommandLine() is called in two places:

CrossTranslationUnitContext::ASTLoader::loadFromSource() keeps the
default parameter value of PrecompilePreambleAfterNParses = 0, meaning
that the preamble file is never created from here.

clang_parseTranslationUnit_Impl(), which passes
CIndexer::PreambleStoragePath to the ASTUnit it creates.

Therefore, the overridden preamble storage path is always used in
TempPCHFile::create().

TempPCHFile::create() uses PreambleStoragePath in the same way as
LibclangInvocationReporter() uses InvocationEmissionPath. The existing
documentation for clang_CXIndex_setInvocationEmissionPathOption() does
not specify ownership, encoding, separator or relative vs absolute path
requirements. So the documentation for
CXIndexOptions::PreambleStoragePath doesn't either. The assumptions are:

no ownership transfer;
UTF-8 encoding;
native separators.
Both relative and absolute paths are supported.

The added API works as expected in KDevelop:
https://invent.kde.org/kdevelop/kdevelop/-/merge_requests/283

Fixes: https://github.com/llvm/llvm-project/issues/51847
Differential Revision: https://reviews.llvm.org/D143418

show more ...

[clang/unittests] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the a

[clang/unittests] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[clang] createInvocationFromCommandLine -> createInvocation, delete former. NFC

(Followup from 40c13720a4b977d4347bbde53c52a4d0703823c2)

Differential Revision: https://reviews.llvm.org/D125012

[clang][Frontend] Add missing error handling

Some early errors during the ASTUnit creation were not transferred to the `FailedParseDiagnostic` so when the code in `LoadFromCommandLine` swaps its con

[clang][Frontend] Add missing error handling

Some early errors during the ASTUnit creation were not transferred to the `FailedParseDiagnostic` so when the code in `LoadFromCommandLine` swaps its content with the content of `StoredDiagnostics` they cannot be retrieved by the user in any way.

Reviewed By: andrewrk, dblaikie

Differential Revision: https://reviews.llvm.org/D78658

show more ...

Correctly set CompilingPCH in PrecompilePreambleAction.

This fixes a crash bug in clangd when used with modules. ASTWriter would
end up writing references to submodules into the PCH file, but upon
r

Correctly set CompilingPCH in PrecompilePreambleAction.

This fixes a crash bug in clangd when used with modules. ASTWriter would
end up writing references to submodules into the PCH file, but upon
reading the submodules would not exists and
HeaderFileInfoTrait::ReadData would crash.

Differential Revision: https://reviews.llvm.org/D85532

show more ...

Avoid including FileManager.h from SourceManager.h

Most clients of SourceManager.h need to do things like turning source
locations into file & line number pairs, but this doesn't require
bringing in

Avoid including FileManager.h from SourceManager.h

Most clients of SourceManager.h need to do things like turning source
locations into file & line number pairs, but this doesn't require
bringing in FileManager.h and LLVM's FS headers.

The main code change here is to sink SM::createFileID into the cpp file.
I reason that this is not performance critical because it doesn't happen
on the diagnostic path, it happens along the paths of macro expansion
(could be hot) and new includes (less hot).

Saves some includes:
309 - /usr/local/google/home/rnk/llvm-project/clang/include/clang/Basic/FileManager.h
272 - /usr/local/google/home/rnk/llvm-project/clang/include/clang/Basic/FileSystemOptions.h
271 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/VirtualFileSystem.h
267 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/FileSystem.h
266 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/Chrono.h

Differential Revision: https://reviews.llvm.org/D75406

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
o

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368942

show more ...

[libclang] Allow skipping warnings from all included files

Depending on the included files and the used warning flags, e.g. -
Weverything, a huge number of warnings can be reported for included
file

[libclang] Allow skipping warnings from all included files

Depending on the included files and the used warning flags, e.g. -
Weverything, a huge number of warnings can be reported for included
files. As processing that many diagnostics comes with a performance
impact and not all clients are interested in those diagnostics, add a
flag to skip them.

Differential Revision: https://reviews.llvm.org/D48116

llvm-svn: 363067

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

Fix build - use llvm::make_unique

llvm-svn: 334072

[Frontend] Honor UserFilesAreVolatile flag getting file buffer in ASTUnit

Do not memory map the main file if the flag UserFilesAreVolatile is set to true
in ASTUnit when calling FileSystem::getBuffe

[Frontend] Honor UserFilesAreVolatile flag getting file buffer in ASTUnit

Do not memory map the main file if the flag UserFilesAreVolatile is set to true
in ASTUnit when calling FileSystem::getBufferForFile.

Differential Revision: https://reviews.llvm.org/D47460

llvm-svn: 334070

show more ...

Commit missing fixes for tool_file_rename

llvm-svn: 314051

[Frontend] Fix printing policy for AST context loaded from file

In ASTUnit::LoadFromASTFile, the context object is set up using
default-constructed LangOptions (which only later get populated). As t

[Frontend] Fix printing policy for AST context loaded from file

In ASTUnit::LoadFromASTFile, the context object is set up using
default-constructed LangOptions (which only later get populated). As the
language options are used in the constructor of PrintingPolicy, this
needs to be updated explicitly after the language options are available.

Patch by Johann Klähn!

Differential Revision: https://reviews.llvm.org/D35271

llvm-svn: 311787

show more ...