Fix tmp files being left on Windows builds.

Clang writes object files by first writing to a .tmp file and then
renaming to the final .obj name. On Windows, if a compile is killed
partway through the

Fix tmp files being left on Windows builds.

Clang writes object files by first writing to a .tmp file and then
renaming to the final .obj name. On Windows, if a compile is killed
partway through the .tmp files don't get deleted.

Currently it seems like RemoveFileOnSignal takes care of deleting the
tmp files on Linux, but on Windows we need to call
setDeleteDisposition on tmp files so that they are deleted when
closed.

This patch switches to using TempFile to create the .tmp files we write
when creating object files, since it uses setDeleteDisposition on Windows.
This change applies to both Linux and Windows for consistency.

Differential Revision: https://reviews.llvm.org/D102876

show more ...

[OpenCL] Add support of OpenCL C 3.0 __opencl_c_fp64

There already exists cl_khr_fp64 extension. So OpenCL C 3.0
and higher should use the feature, earlier versions still
use the extension. OpenCL C

[OpenCL] Add support of OpenCL C 3.0 __opencl_c_fp64

There already exists cl_khr_fp64 extension. So OpenCL C 3.0
and higher should use the feature, earlier versions still
use the extension. OpenCL C 3.0 API spec states that extension
will be not described in the option string if corresponding
optional functionality is not supported (see 4.2. Querying Devices).
Due to that fact the usage of features for OpenCL C 3.0 must
be as follows:

```
$ clang -Xclang -cl-ext=+cl_khr_fp64,+__opencl_c_fp64 ...

$ clang -Xclang -cl-ext=-cl_khr_fp64,-__opencl_c_fp64 ...
```

e.g. the feature and the equivalent extension (if exists)
must be set to the same values

Reviewed By: Anastasia

Differential Revision: https://reviews.llvm.org/D96524

show more ...

Modules: Simplify how DisableGeneratingGlobalModuleIndex is set, likely NFC

DisableGeneratingGlobalModuleIndex was being set by
CompilerInstance::findOrCompileModuleAndReadAST most of (but not all o

Modules: Simplify how DisableGeneratingGlobalModuleIndex is set, likely NFC

DisableGeneratingGlobalModuleIndex was being set by
CompilerInstance::findOrCompileModuleAndReadAST most of (but not all of)
the times it returned `nullptr` as a "normal" failure. Pull that up to
the caller, CompilerInstance::loadModule, to simplify the code. This
resolves a number of FIXMEs added during the refactoring in
5cca622310c10fdf6f921b6cce26f91d9f14c762.

The extra cases where this is set are all some version of a fatal error,
and the only client of the field, shouldBuildGlobalModuleIndex, seems
to be unreachable in that case. Even if there is some corner case where
this has an effect, it seems like the right/consistent behaviour.

Differential Revision: https://reviews.llvm.org/D101672

show more ...

Modules: Rename ModuleBuildFailed => DisableGeneratingGlobalModuleIndex, NFC

Rename CompilerInstance's ModuleBuildFailed field to
DisableGeneratingGlobalModuleIndex, which more precisely describes i

Modules: Rename ModuleBuildFailed => DisableGeneratingGlobalModuleIndex, NFC

Rename CompilerInstance's ModuleBuildFailed field to
DisableGeneratingGlobalModuleIndex, which more precisely describes its
role. Otherwise, it's hard to suss out how it's different from
ModuleLoader::HadFatalFailure, and what sort of code simplifications are
safe.

Differential Revision: https://reviews.llvm.org/D101670

show more ...

Modules: Remove ModuleLoader::OtherUncachedFailure, NFC

5cca622310c10fdf6f921b6cce26f91d9f14c762 refactored
CompilerInstance::loadModule, splitting out
findOrCompileModuleAndReadAST, but was careful

Modules: Remove ModuleLoader::OtherUncachedFailure, NFC

5cca622310c10fdf6f921b6cce26f91d9f14c762 refactored
CompilerInstance::loadModule, splitting out
findOrCompileModuleAndReadAST, but was careful to avoid making any
functional changes. It added ModuleLoader::OtherUncachedFailure to
facilitate this and left behind FIXMEs asking why certain failures
weren't cached.

After a closer look, I think we can just remove this and simplify the
code. This changes the behaviour of the following (simplified) code from
CompilerInstance::loadModule, causing a failure to be cached more often:

```
if (auto MaybeModule = MM.getCachedModuleLoad(*Path[0].first))
return *MaybeModule;
if (ModuleName == getLangOpts().CurrentModule)
return MM.cacheModuleLoad(PP.lookupModule(...));
ModuleLoadResult Result = findOrCompileModuleAndReadAST(...);
if (Result.isNormal()) // This will be 'true' more often.
return MM.cacheModuleLoad(..., Module);
return Result;
```

`MM` here is a ModuleMap owned by the Preprocessor. Here are the cases
where `findOrCompileModuleAndReadAST` starts returning a "normal" failed
result:
- Emitted `diag::err_module_not_found`, where there's no module map
found.
- Emitted `diag::err_module_build_disabled`, where implicitly building
modules is disabled.
- Emitted `diag::err_module_cycle`, which detects module cycles in the
implicit modules build system.
- Emitted `diag::err_module_not_built`, which avoids building a module
in this CompilerInstance if another one tried and failed already.
- `compileModuleAndReadAST()` was called and failed to build.

The four errors are all fatal, and last item also reports a fatal error,
so it this extra caching has no functionality change... but even if it
did, it seems fine to cache these failed results within a ModuleMap
instance (note that each CompilerInstance has its own Preprocessor and
ModuleMap).

Differential Revision: https://reviews.llvm.org/D101667

show more ...

Modules: Remove an extra early return, NFC

Remove an early return from an `else` block that's immediately followed
by an equivalent early return after the `else` block.

Differential Revision: https

Modules: Remove an extra early return, NFC

Remove an early return from an `else` block that's immediately followed
by an equivalent early return after the `else` block.

Differential Revision: https://reviews.llvm.org/D101671

show more ...

[OpenCL] Introduce new method for validating OpenCL target

Language options are not available when a target is being created,
thus, a new method is introduced. Also, some refactoring is done,
such a

[OpenCL] Introduce new method for validating OpenCL target

Language options are not available when a target is being created,
thus, a new method is introduced. Also, some refactoring is done,
such as removing OpenCL feature macros setting from TargetInfo.

Reviewed By: Anastasia

Differential Revision: https://reviews.llvm.org/D101087

show more ...

[clang] Check AuxTarget exists when creating target in CompilerInstance

D97493 separate target creation out to a single function
`CompilerInstance::createTarget`. However, it would overwrite AuxTarg

[clang] Check AuxTarget exists when creating target in CompilerInstance

D97493 separate target creation out to a single function
`CompilerInstance::createTarget`. However, it would overwrite AuxTarget
even if it has been set.
As @kadircet recommended in D98128, this patch check the existence of
AuxTarget and not overwrite it when it has been set.

Reviewed By: kadircet

Differential Revision: https://reviews.llvm.org/D100024

show more ...

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a C

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.

Solution:
This patch adds two new flags

- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.

Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.

So this is the behaviour per platform with my patch:

z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode

Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return

The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```

These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D99426

show more ...

[SystemZ][z/OS] Distinguish between text and binary files on z/OS

This patch consists of the initial changes to help distinguish between text and binary content correctly on z/OS. I would like to ge

[SystemZ][z/OS] Distinguish between text and binary files on z/OS

This patch consists of the initial changes to help distinguish between text and binary content correctly on z/OS. I would like to get feedback from Windows users on setting OF_None for all ToolOutputFiles. This seems to have been done as an optimization to prevent CRLF translation on Windows in the past.

Reviewed By: zibi

Differential Revision: https://reviews.llvm.org/D97785

show more ...

[clang][NFC] Extract Target and AuxTarget creation in CompilerInstance to new function

As @sammccall mentioned in [[ https://reviews.llvm.org/D97109 | D97109 ]], I've extract the logic of creating T

[clang][NFC] Extract Target and AuxTarget creation in CompilerInstance to new function

As @sammccall mentioned in [[ https://reviews.llvm.org/D97109 | D97109 ]], I've extract the logic of creating Target and AuxTarget into a new function called `createTargetAndAuxTarget`.

Since there are many similar code in clang or other related tools, consolidating them into a single function may help others to maintain the logic handling target related things.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D97493

show more ...

Make sure a module file with errors produced via '-fallow-pcm-with-compiler-errors' can be loaded when using implicit modules

A module with errors would be marked as out-of-date, then the `compilerM

Make sure a module file with errors produced via '-fallow-pcm-with-compiler-errors' can be loaded when using implicit modules

A module with errors would be marked as out-of-date, then the `compilerModule` action would produce it, but due to the error it would be treated as failure and the resulting PCM would not get used.

rdar://74087062

Differential Revision: https://reviews.llvm.org/D96246

show more ...

Frontend: Use early returns in CompilerInstance::clearOutputFiles, NFC

Use early returns in `CompilerInstance::clearOutputFiles` to clarify the
logic, and rename `ec` to `EC` as a drive-by.

No func

Frontend: Use early returns in CompilerInstance::clearOutputFiles, NFC

Use early returns in `CompilerInstance::clearOutputFiles` to clarify the
logic, and rename `ec` to `EC` as a drive-by.

No functionality change.

show more ...

Frontend: Fix layering between create{,Default}OutputFile, NFC

Fix layering between `CompilerInstance::createDefaultOutputFile` and the
two versions of `createOutputFile`.

- Add missing configurati

Frontend: Fix layering between create{,Default}OutputFile, NFC

Fix layering between `CompilerInstance::createDefaultOutputFile` and the
two versions of `createOutputFile`.

- Add missing configuration flags to `createDefaultOutputFile` so that
GeneratePCHAction and GenerateModuleFromModuleMapAction can use it.
They previously promised that temporary files were turned on; now
`createDefaultOutputFile` handles that logic.
- Lift the logic handling `InFile` and `Extension` to
`createDefaultOutputFile`, since it's only the callers of that
function that are using it.
- Rename the deeper of the two `createOutputFile`s to
`createOutputFileImpl` and make it private to `CompilerInstance` (to
prove that no one else is using it).
- Sink the logic for adding to `CompilerInstance::OutputFiles` down to
`createOutputFileImpl`, allowing two "optional" (but always used)
`std::string*` out parameters to be removed.
- Instead of passing a `std::error_code` out parameter into
`createOutputFileImpl`, have it return `Expected<>`.
- As a drive-by, inline `CompilerInstance::addOutputFile` into its only
caller, `createOutputFileImpl`.

Clean layering makes it easier for a future commit to extract
`createOutputFileImpl` out of `CompilerInstance`.

Differential Revision: https://reviews.llvm.org/D93248

show more ...

Frontend: Simplify handling of non-seeking streams in CompilerInstance, NFC

Add a new `raw_pwrite_ostream` variant, `buffer_unique_ostream`, which
is like `buffer_ostream` but with unique ownership

Frontend: Simplify handling of non-seeking streams in CompilerInstance, NFC

Add a new `raw_pwrite_ostream` variant, `buffer_unique_ostream`, which
is like `buffer_ostream` but with unique ownership of the stream it's
wrapping. Use this in CompilerInstance to simplify the ownership of
non-seeking output streams, avoiding logic sprawled around to deal with
them specially.

This also simplifies future work to encapsulate output files in a
different class.

Differential Revision: https://reviews.llvm.org/D93260

show more ...

Frontend: Fix memory leak in CompilerInstance::setVerboseOutputStream

Found this memory leak in `CompilerInstance::setVerboseOutputStream` by
inspection; it looks like this wasn't previously exercis

Frontend: Fix memory leak in CompilerInstance::setVerboseOutputStream

Found this memory leak in `CompilerInstance::setVerboseOutputStream` by
inspection; it looks like this wasn't previously exercised, since it was
never called twice.

Differential Revision: https://reviews.llvm.org/D93249

show more ...

[ASTReader] Allow controlling separately whether validation should be disabled for a PCH vs a module file

This addresses an issue with how the PCH preable works, specifically:

1. When using a PCH/p

[ASTReader] Allow controlling separately whether validation should be disabled for a PCH vs a module file

This addresses an issue with how the PCH preable works, specifically:

1. When using a PCH/preamble the module hash changes and a different cache directory is used
2. When the preamble is used, PCH & PCM validation is disabled.

Due to combination of #1 and #2, reparsing with preamble enabled can end up loading a stale module file before a header change and using it without updating it because validation is disabled and it doesn’t check that the header has changed and the module file is out-of-date.

rdar://72611253

Differential Revision: https://reviews.llvm.org/D95159

show more ...

Basic: Add native support for stdin to SourceManager and FileManager

Add support for stdin to SourceManager and FileManager. Adds
FileManager::getSTDIN, which adds a FileEntryRef for `<stdin>` and r

Basic: Add native support for stdin to SourceManager and FileManager

Add support for stdin to SourceManager and FileManager. Adds
FileManager::getSTDIN, which adds a FileEntryRef for `<stdin>` and reads
the MemoryBuffer, which is stored as `FileEntry::Content`.

Eventually the other buffers in `ContentCache` will sink to here as well
-- we probably usually want to load/save a MemoryBuffer eagerly -- but
it's happening early for stdin to get rid of
CompilerInstance::InitializeSourceManager's final call to
`SourceManager::overrideFileContents`.

clang/test/CXX/modules-ts/dcl.dcl/dcl.module/dcl.module.export/p1.cpp
relies on building a module from stdin; supporting that requires setting
ContentCache::BufferOverridden.

Differential Revision: https://reviews.llvm.org/D93148

show more ...

Basic: Support named pipes natively in SourceManager and FileManager

Handle named pipes natively in SourceManager and FileManager, removing a
call to `SourceManager::overrideFileContents` in
`Compil

Basic: Support named pipes natively in SourceManager and FileManager

Handle named pipes natively in SourceManager and FileManager, removing a
call to `SourceManager::overrideFileContents` in
`CompilerInstance::InitializeSourceManager` (removing a blocker for
sinking the content cache to FileManager (which will incidently sink
this new named pipe logic with it)).

SourceManager usually checks if the file entry's size matches the
eventually loaded buffer, but that's now skipped for named pipes since
the `stat` won't reflect the full size. Since we can't trust
`ContentsEntry->getSize()`, we also need shift the check for files that
are too large until after the buffer is loaded... and load the buffer
immediately in `createFileID` so that no client gets a bad value from
`ContentCache::getSize`. `FileManager::getBufferForFile` also needs to
treat these files as volatile when loading the buffer.

Native support in SourceManager / FileManager means that named pipes can
also be `#include`d, and clang/test/Misc/dev-fd-fs.c was expanded to
check for that.

This is a new version of 3b18a594c7717a328c33b9c1eba675e9f4bd367c, which
was reverted in b34632201987eed369bb7ef4646f341b901c95b8 since it was
missing the `SourceManager` changes.

Differential Revision: https://reviews.llvm.org/D92531

show more ...

Frontend: Fix confusing comment at call to clearOutputFiles, NFC

Fix the comment in front of `compileModuleImpl`'s call to
`CompilerInstance::clearOutputFiles`. The purpose of this call is to
delete

Frontend: Fix confusing comment at call to clearOutputFiles, NFC

Fix the comment in front of `compileModuleImpl`'s call to
`CompilerInstance::clearOutputFiles`. The purpose of this call is to
delete any stray temporary files after the module generation thread
crashes.

The comment is from f545f67de3a1bfdbbfad88acde5b540ce3b82f4f, and
was associated with manually deleting a generated module map. Then
13afbf42d830dd43febbeb0855aa359ca9dbfbf9 added this `clearOutputFiles`
call between the comment and the code it referenced. Finally,
1f76c4e8101b9beaf8f1b10a57faa80256ab2b05 started sending the generated
module map directly to the SourceManager instead of putting it on disk,
deleting the call that the comment referenced.

No functionality change.

show more ...

Frontend: Use a getVirtualFileRef for a named pipe main file, NFC

Frontend: Migrate to FileEntryRef in CompilerInstance::InitializeSourceManager, NFC

Use `FileManager::getVirtualFileRef` to get the virtual file for stdin,
and add an overload of `SourceManager::ove

Frontend: Migrate to FileEntryRef in CompilerInstance::InitializeSourceManager, NFC

Use `FileManager::getVirtualFileRef` to get the virtual file for stdin,
and add an overload of `SourceManager::overrideFileContents` that takes
a `FileEntryRef`, migrating `CompilerInstance::InitializeSourceManager`.

Differential Revision: https://reviews.llvm.org/D92680

show more ...

[Time-report] Add a flag -ftime-report={per-pass,per-pass-run} to control the pass timing aggregation

Currently, -ftime-report + new pass manager emits one line of report for each
pass run. This pot

[Time-report] Add a flag -ftime-report={per-pass,per-pass-run} to control the pass timing aggregation

Currently, -ftime-report + new pass manager emits one line of report for each
pass run. This potentially causes huge output text especially with regular LTO
or large single file (Obeserved in private tests and was reported in D51276).
The behaviour of -ftime-report + legacy pass manager is
emitting one line of report for each pass object which has relatively reasonable
text output size. This patch adds a flag `-ftime-report=` to control time report
aggregation for new pass manager.

The flag is for new pass manager only. Using it with legacy pass manager gives
an error. It is a driver and cc1 flag. `per-pass` is the new default so
`-ftime-report` is aliased to `-ftime-report=per-pass`. Before this patch,
functionality-wise `-ftime-report` is aliased to `-ftime-report=per-pass-run`.

* Adds an boolean variable TimePassesHandler::PerRun to control per-pass vs per-pass-run.
* Adds a new clang CodeGen flag CodeGenOptions::TimePassesPerRun to work with the existing CodeGenOptions::TimePasses.
* Remove FrontendOptions::ShowTimers, its uses are replaced by the existing CodeGenOptions::TimePasses.
* Remove FrontendTimesIsEnabled (It was introduced in D45619 which was largely reverted.)

Differential Revision: https://reviews.llvm.org/D92436

show more ...

Revert "Frontend: Sink named pipe logic from CompilerInstance down to FileManager"

This reverts commit 3b18a594c7717a328c33b9c1eba675e9f4bd367c, since
apparently this doesn't work everywhere. E.g.,

Revert "Frontend: Sink named pipe logic from CompilerInstance down to FileManager"

This reverts commit 3b18a594c7717a328c33b9c1eba675e9f4bd367c, since
apparently this doesn't work everywhere. E.g.,
clang-x86_64-debian-fast/3889
(http://lab.llvm.org:8011/#/builders/109/builds/3889) gives me:
```
+ : 'RUN: at line 8'
+ /b/1/clang-x86_64-debian-fast/llvm.obj/bin/clang -x c /dev/fd/0 -E
+ cat /b/1/clang-x86_64-debian-fast/llvm.src/clang/test/Misc/dev-fd-fs.c
fatal error: file '/dev/fd/0' modified since it was first processed
1 error generated.
```

show more ...

Frontend: Sink named pipe logic from CompilerInstance down to FileManager

Remove compilicated logic from CompilerInstance::InitializeSourceManager
to deal with named pipes, updating FileManager::get

Frontend: Sink named pipe logic from CompilerInstance down to FileManager

Remove compilicated logic from CompilerInstance::InitializeSourceManager
to deal with named pipes, updating FileManager::getBufferForFile to
handle it in a more straightforward way. The existing test at
clang/test/Misc/dev-fd-fs.c covers the new behaviour (just like it did
the old behaviour).

Differential Revision: https://reviews.llvm.org/D90733

show more ...