Add -frewrite-imports flag.

If specified, when preprocessing, the contents of imported .pcm files will be
included in preprocessed output. The resulting preprocessed file can then be
compiled standa

Add -frewrite-imports flag.

If specified, when preprocessing, the contents of imported .pcm files will be
included in preprocessed output. The resulting preprocessed file can then be
compiled standalone without the module sources or .pcm files.

llvm-svn: 305116

show more ...

Add #pragma clang module build/endbuild pragmas for performing a module build
as part of a compilation.

This is intended for two purposes:

1) Writing self-contained test cases for modules: we can n

Add #pragma clang module build/endbuild pragmas for performing a module build
as part of a compilation.

This is intended for two purposes:

1) Writing self-contained test cases for modules: we can now write a single
source file test that builds some number of module files on the side and
imports them.

2) Debugging / test case reduction. A single-source testcase is much more
amenable to reduction, compared to a VFS tarball or .pcm files.

llvm-svn: 305101

show more ...

Factor out and unify emission of "module is unavailable" diagnostics.

Inspired by post-commit review of r304190.

llvm-svn: 304728

Added LLVM_FALLTHROUGH to address warning: this statement may fall through. NFC.

llvm-svn: 304645

Move functionality for handling module maps as inputs from the -emit-module
action to the general FrontendAction infrastructure.

This permits applying -E, -ast-dump, -fsyntax-only, and so on to a mo

Move functionality for handling module maps as inputs from the -emit-module
action to the general FrontendAction infrastructure.

This permits applying -E, -ast-dump, -fsyntax-only, and so on to a module map
compilation. (The -E form is not currently especially useful yet as there's no
good way to take the output and use it to actually build a module.)

In order to support this, -cc1 now accepts -x <lang>-module-map in all cases
where it accepts -x <lang> for a language we can parse (not ir/ast). And for
uniformity, we also accept -x <lang>-header for all such languages (we used
to reject for cuda and renderscript), and -x <lang>-cpp-output for all such
languages (we used to reject for c, cl, and renderscript).

(None of these new alternatives are accepted by the driver yet, so no
user-visible changes.)

llvm-svn: 301610

show more ...

Preprocessor: Suppress -Wnonportable-include-path for header maps

If a file search involves a header map, suppress
-Wnonportable-include-path. It's firing lots of false positives for
framework auth

Preprocessor: Suppress -Wnonportable-include-path for header maps

If a file search involves a header map, suppress
-Wnonportable-include-path. It's firing lots of false positives for
framework authors internally, and it's not trivial to fix.

Consider a framework called "Foo" with a main (installed) framework header
"Foo/Foo.h". It's atypical for "Foo.h" to actually live inside a
directory called "Foo" in the source repository. Instead, the
build system generates a header map while building the framework.
If Foo.h lives at the top-level of the source repository (common), and
the git repo is called ssh://some.url/foo.git, then the header map will
have something like:

Foo/Foo.h -> /Users/myname/code/foo/Foo.h

where "/Users/myname/code/foo" is the clone of ssh://some.url/foo.git.

After #import <Foo/Foo.h>, the current implementation of
-Wnonportable-include-path will falsely assume that Foo.h was found in a
nonportable way, because of the name of the git clone (.../foo/Foo.h).
However, that directory name was not involved in the header search at
all.

This commit adds an extra parameter to Preprocessor::LookupFile and
HeaderSearch::LookupFile to track if the search used a header map,
making it easy to suppress the warning. Longer term, once we find a way
to avoid the false positive, we should turn the warning back on.

rdar://problem/28863903

llvm-svn: 301592

show more ...

Refactor frontend InputKind to prepare for treating module maps as a distinct kind of input.

No functionality change intended.

llvm-svn: 301442

[PCH] Attach instance's dependency collectors to PCH external AST sources.

Summary:
When a PCH is included via -include-pch, clang should treat the
current TU as dependent on the sourcefile that the

[PCH] Attach instance's dependency collectors to PCH external AST sources.

Summary:
When a PCH is included via -include-pch, clang should treat the
current TU as dependent on the sourcefile that the PCH was generated from.

This is currently _partly_ accomplished by InitializePreprocessor calling
AddImplicitIncludePCH to synthesize an implicit #include of the sourcefile,
into the preprocessor's Predefines buffer.

For FrontendActions such as PreprocessOnlyAction (which is, curiously, what the
driver winds up running one of in response to a plain clang -M) this is
sufficient: the preprocessor cranks over its Predefines and emits a dependency
reference to the initial sourcefile.

For other FrontendActions (for example -emit-obj or -fsyntax-only) the
Predefines buffer is reset to the suggested predefines buffer from the PCH, so
the dependency edge is lost. The result is that clang emits a .d file in those
cases that lacks a reference to the .h file responsible for the input (and in
Swift's case, our .swiftdeps file winds up not including a reference to the
source file for a PCH bridging header.)

This patch fixes the problem by taking a different tack: ignoring the
Predefines buffer (which seems a bit like a hack anyways) and directly
attaching the CompilerInstance's DependencyCollectors (and legacy
DependencyFileGenerator) to the ASTReader for the external AST.

This approach is similar to the one chosen in earlier consultation with Bruno
and Ben, and I think it's the least-bad solution, given several options.

Reviewers: bruno, benlangmuir, doug.gregor

Reviewed By: bruno, doug.gregor

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D31378

llvm-svn: 299009

show more ...

Move setting of LangOpts based on target flags out of CompilerInstance
and into TargetInfo::adjust so that it gets called in more places
throughout the compiler (AST serialization in particular).

Sh

Move setting of LangOpts based on target flags out of CompilerInstance
and into TargetInfo::adjust so that it gets called in more places
throughout the compiler (AST serialization in particular).

Should fix PPC modules after removing of faltivec.

llvm-svn: 298487

show more ...

Remove the -faltivec alias option and replace it with -maltivec everywhere.
The alias was only ever used on darwin and had some issues there,
and isn't used in practice much. Also fixes a problem wit

Remove the -faltivec alias option and replace it with -maltivec everywhere.
The alias was only ever used on darwin and had some issues there,
and isn't used in practice much. Also fixes a problem with -mno-altivec
not turning off -maltivec.

Also add a diagnostic for faltivec/fno-altivec that directs users to use
maltivec options and include the altivec.h file explicitly.

llvm-svn: 298449

show more ...

Reapply "Modules: Cache PCMs in memory and avoid a use-after-free"

This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338). The memory buffer generato

Reapply "Modules: Cache PCMs in memory and avoid a use-after-free"

This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338). The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.

Original commit message follows:

----

Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).

This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.

This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.

The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.

- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.

- When modules fail to validate, they're removed from the cache.

- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.

- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.

Original patch by Manman Ren; most testcases by Adrian Prantl!

llvm-svn: 298278

show more ...

Revert "Modules: Cache PCMs in memory and avoid a use-after-free"

This reverts commit r298165, as it broke the ARM builds.

llvm-svn: 298185

[Modules] In case of lock timeout, fallback and build module

Duncan's r298165 introduced the PCMCache mechanism, which guarantees
that locks aren't necessary anymore for correctness but only for
per

[Modules] In case of lock timeout, fallback and build module

Duncan's r298165 introduced the PCMCache mechanism, which guarantees
that locks aren't necessary anymore for correctness but only for
performance, by avoiding building it twice when possible.

Change the logic to avoid an error but actually build the module in case
the timeout happens. Instead of an error, still emit a remark for
debugging purposes.

rdar://problem/30297862

llvm-svn: 298175

show more ...

Modules: Cache PCMs in memory and avoid a use-after-free

Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back

Modules: Cache PCMs in memory and avoid a use-after-free

Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).

This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.

This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.

The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.

- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.

- When modules fail to validate, they're removed from the cache.

- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.

- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.

Original patch by Manman Ren; most testcases by Adrian Prantl!

llvm-svn: 298165

show more ...

Modules: Use hash of PCM content for SIGNATURE

Change ASTFileSignature from a random 32-bit number to the hash of the
PCM content.

- Move definition ASTFileSignature to Basic/Module.h so Module a

Modules: Use hash of PCM content for SIGNATURE

Change ASTFileSignature from a random 32-bit number to the hash of the
PCM content.

- Move definition ASTFileSignature to Basic/Module.h so Module and
ASTSourceDescriptor can use it.

- Change the signature from uint64_t to std::array<uint32_t,5>.

- Stop using (saving/reading) the size and modification time of PCM
files when there is a valid SIGNATURE.

- Add UNHASHED_CONTROL_BLOCK, and use it to store the SIGNATURE record
and other records that shouldn't affect the hash. Because implicit
modules reuses the same file for multiple levels of -Werror, this
includes DIAGNOSTIC_OPTIONS and DIAG_PRAGMA_MAPPINGS.

This helps to solve a PCH + implicit Modules dependency issue: PCH files
are handled by the external build system, whereas implicit modules are
handled by internal compiler build system. This prevents invalidating a
PCH when the compiler overwrites a PCM file with the same content
(modulo the diagnostic differences).

Design and original patch by Manman Ren!

llvm-svn: 297655

show more ...

Modules: Simplify CompilerInstance constructor, NFC

Initialize fields directly in header. Note that the ModuleManager field is an
IntrusiveRefCntPtr, so there's no need for explicit initialization.

Modules: Simplify CompilerInstance constructor, NFC

Initialize fields directly in header. Note that the ModuleManager field is an
IntrusiveRefCntPtr, so there's no need for explicit initialization.

llvm-svn: 293863

show more ...

Reapply "IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer"

Aleksey Shlypanikov pointed out my mistake in migrating an explicit
unique_ptr to auto - I was exp

Reapply "IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer"

Aleksey Shlypanikov pointed out my mistake in migrating an explicit
unique_ptr to auto - I was expecting the function returned a unique_ptr,
but instead it returned a raw pointer - introducing a leak.

Thanks Aleksey!

This reapplies r291184, reverted in r291249.

llvm-svn: 291270

show more ...

Revert "IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer"

Caused a memory leak reported by asan. Reverting while I investigate.

This reverts commit r291184.

Revert "IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer"

Caused a memory leak reported by asan. Reverting while I investigate.

This reverts commit r291184.

llvm-svn: 291249

show more ...

shared_ptrify (from InclusiveRefCntPtr) HeaderSearchOptions

llvm-svn: 291202

IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer

llvm-svn: 291184

Move Preprocessor over to std::shared_ptr rather than IntrusiveRefCntPtr

llvm-svn: 291166

Move PreprocessorOptions to std::shared_ptr from IntrusiveRefCntPtr

llvm-svn: 291160

Move FailedModulesSet over to shared_ptr from IntrusiveRefCntPtr

llvm-svn: 291159

Use shared_ptr instead of IntrusiveRefCntPtr for ModuleFileExtension

The intrusiveness wasn't needed here, so this simplifies/clarifies the
ownership model.

llvm-svn: 291150

[CrashReproducer] Add support for merging -ivfsoverlay

Merge all VFS mapped files inside -ivfsoverlay inputs into the vfs
overlay provided by the crash reproducer. This is the last missing piece
to

[CrashReproducer] Add support for merging -ivfsoverlay

Merge all VFS mapped files inside -ivfsoverlay inputs into the vfs
overlay provided by the crash reproducer. This is the last missing piece
to allow crash reproducers to fully work with user frameworks; when
combined with headermaps, it allows clang to find additional frameworks.

rdar://problem/27913709

llvm-svn: 290326

show more ...