Revert "Invalidate the file system cache entries for files that may rebuild"

This reverts commit r209910, which is breaking some of the bots.

llvm-svn: 209911

Invalidate the file system cache entries for files that may rebuild

I thought we could get away without this, but it means that the
FileEntry objects actually refer to the wrong files, since pcms ar

Invalidate the file system cache entries for files that may rebuild

I thought we could get away without this, but it means that the
FileEntry objects actually refer to the wrong files, since pcms are not
updated inplace, they are atomically renamed into place after compiling
a module.

So we are close to the original behaviour of invalidating the cache for
all modules being removed, but now we should only invalidate the ones
that depend on whichever module failed to load.

Unfortunately I haven't come up with a new test that didn't require
a race between parallel invocations of clang.

<rdar://problem/17038180>

llvm-svn: 209910

show more ...

[C++11] Use 'nullptr'. Serialization edition.

llvm-svn: 209392

Speculative fix for Windows buildbot after r209138

It appears that Windows doesn't like renaming over open files, which we
do in clearOutputFiles. The file being compiled should be safe to
removed,

Speculative fix for Windows buildbot after r209138

It appears that Windows doesn't like renaming over open files, which we
do in clearOutputFiles. The file being compiled should be safe to
removed, but this isn't very satisfying - we don't want to manually
manage the lifetime of files we cannot prove have no references.

llvm-svn: 209195

show more ...

Don't refresh stat() info for pcm files

Follow-up fix for 209138. Actually, since we already have this file
open, we don't want to refresh the stat() info, since that might be
newer than what we ha

Don't refresh stat() info for pcm files

Follow-up fix for 209138. Actually, since we already have this file
open, we don't want to refresh the stat() info, since that might be
newer than what we have open (bad!).

llvm-svn: 209143

show more ...

Fix use-after-free and spurious error during module load

FileManager::invalidateCache is not safe to call when there may be
existing references to the file. What module load failure needs is
to refr

Fix use-after-free and spurious error during module load

FileManager::invalidateCache is not safe to call when there may be
existing references to the file. What module load failure needs is
to refresh so stale stat() info isn't stored.

This may be the last user of invalidateCache; I'll take a look and
remove it if possible in a future commit.

This caused a use-after-free error as well as a spurious error message
that a module was "found in both 'X.pcm' and 'X.pcm'" in some cases.

llvm-svn: 209138

show more ...

Fix a use-after-free bug I recently introduced in lookupModuleFile

llvm-svn: 207932

Avoid a potential race between stat() and open() of ASTFile

We need to open an ASTFile while checking its expected size and
modification time, or another clang instance can modify the file between
t

Avoid a potential race between stat() and open() of ASTFile

We need to open an ASTFile while checking its expected size and
modification time, or another clang instance can modify the file between
the stat() and the open().

llvm-svn: 207735

show more ...

Allow multiple modules with the same name to coexist in the module cache

To differentiate between two modules with the same name, we will
consider the path the module map file that they are defined

Allow multiple modules with the same name to coexist in the module cache

To differentiate between two modules with the same name, we will
consider the path the module map file that they are defined by* part of
the ‘key’ for looking up the precompiled module (pcm file).
Specifically, this patch renames the precompiled module (pcm) files from
cache-path/<module hash>/Foo.pcm
to
cache-path/<module hash>/Foo-<hash of module map path>.pcm

In addition, I’ve taught the ASTReader to re-resolve the names of
imported modules during module loading so that if the header search
context changes between when a module was originally built and when it
is loaded we can rebuild it if necessary. For example, if module A
imports module B

first time:
clang -I /path/to/A -I /path/to/B ...

second time:
clang -I /path/to/A -I /different/path/to/B ...

will now rebuild A as expected.

* in the case of inferred modules, we use the module map file that
allowed the inference, not the __inferred_module.map file, since the
inferred file path is the same for every inferred module.

llvm-svn: 206201

show more ...

[C++11] Remove a now unnecessary use of std::function for a remove_if
predicate. The wrapper used by SetVector was erroneously requiring an
adaptable predicate. It has been fixed and we really don't

[C++11] Remove a now unnecessary use of std::function for a remove_if
predicate. The wrapper used by SetVector was erroneously requiring an
adaptable predicate. It has been fixed and we really don't want to
require an indirect call for every predicate evaluation.

llvm-svn: 202744

show more ...

[C++11] Replace verbose functors with succinct lambdas

No functionality change.

llvm-svn: 202590

Recommit virtual file system

Previously reverted in r201755 due to causing an assertion failure.

I've removed the offending assertion, and taught the CompilerInstance to
create a default virtual fi

Recommit virtual file system

Previously reverted in r201755 due to causing an assertion failure.

I've removed the offending assertion, and taught the CompilerInstance to
create a default virtual file system inside createFileManager. In the
future, we should be able to reach into the CompilerInvocation to
customize this behaviour without breaking clients that don't care.

llvm-svn: 201818

show more ...

Reverting the virtual file system implementation, because it triggers an assertion
in our internal build bots.

This reverts commits 201618, 201635, 201636, 201639, 201685, 201691, and 201696.

llvm-

Reverting the virtual file system implementation, because it triggers an assertion
in our internal build bots.

This reverts commits 201618, 201635, 201636, 201639, 201685, 201691, and 201696.

llvm-svn: 201755

show more ...

Initial implementation of virtual file system

This adds the minimum virtual file system support to start migrating
FileManager onto the VFS.

Originally discussed here:
http://lists.cs.uiuc.edu/pipe

Initial implementation of virtual file system

This adds the minimum virtual file system support to start migrating
FileManager onto the VFS.

Originally discussed here:
http://lists.cs.uiuc.edu/pipermail/cfe-dev/2014-February/035188.html

Differential Revision: http://llvm-reviews.chandlerc.com/D2745

llvm-svn: 201618

show more ...

Add an option to allow Clang verify source files for a module only once during
the build

When Clang loads the module, it verifies the user source files that the module
was built from. If any file w

Add an option to allow Clang verify source files for a module only once during
the build

When Clang loads the module, it verifies the user source files that the module
was built from. If any file was changed, the module is rebuilt. There are two
problems with this:
1. correctness: we don't verify system files (there are too many of them, and
stat'ing all of them would take a lot of time);
2. performance: the same module file is verified again and again during a
single build.

This change allows the build system to optimize source file verification. The
idea is based on the fact that while the project is being built, the source
files don't change. This allows us to verify the module only once during a
single build session. The build system passes a flag,
-fbuild-session-timestamp=, to inform Clang of the time when the build started.
The build system also requests to enable this feature by passing
-fmodules-validate-once-per-build-session. If these flags are not passed, the
behavior is not changed. When Clang verifies the module the first time, it
writes out a timestamp file. Then, when Clang loads the module the second
time, it finds a timestamp file, so it can compare the verification timestamp
of the module with the time when the build started. If the verification
timestamp is too old, the module is verified again, and the timestamp file is
updated.

llvm-svn: 201224

show more ...

Add a bit more info to modules fatal error.

Just a minor tweak to make it easier to track down the cause of fatal errors
with modules.

llvm-svn: 190108

This wasn't headers, just missing namespaces.

/me bows head in shame.

llvm-svn: 189172

Add missing includes.

llvm-svn: 189171

Use pop_back_val() instead of both back() and pop_back().
No functionality change intended.

llvm-svn: 189112

Include Path.h instead of PathV2.h.

I am about to move PathV2.h to Path.h.

llvm-svn: 183795

<rdar://problem/13509689> Introduce -module-file-info option that provides information about a particular module file.

This option can be useful for end users who want to know why they
ended up with

<rdar://problem/13509689> Introduce -module-file-info option that provides information about a particular module file.

This option can be useful for end users who want to know why they
ended up with a ton of different variants of the "std" module in their
module cache. This problem should go away over time, as we reduce the
need for module variants, but it will never go away entirely.

llvm-svn: 178148

show more ...

<rdar://problem/13479539> Simplify ModuleManager/GlobalModuleIndex interaction to eliminate a pile of extraneous stats().

The refactoring in r177367 introduced a serious performance bug where
the "l

<rdar://problem/13479539> Simplify ModuleManager/GlobalModuleIndex interaction to eliminate a pile of extraneous stats().

The refactoring in r177367 introduced a serious performance bug where
the "lazy" resolution of module file names in the global module index
to actual module file entries in the module manager would perform
repeated negative stats(). The new interaction requires the module
manager to inform the global module index when a module file has been
loaded, eliminating the extraneous stat()s and a bunch of bookkeeping
on both sides.

llvm-svn: 177750

show more ...

Remove unused variable.

llvm-svn: 177657

<rdar://problem/13363214> Eliminate race condition between module rebuild and the global module index.

The global module index was querying the file manager for each of the
module files it knows abo

<rdar://problem/13363214> Eliminate race condition between module rebuild and the global module index.

The global module index was querying the file manager for each of the
module files it knows about at load time, to prune out any out-of-date
information. The file manager would then cache the results of the
stat() falls used to find that module file.

Later, the same translation unit could end up trying to import one of the
module files that had previously been ignored by the module cache, but
after some other Clang instance rebuilt the module file to bring it
up-to-date. The stale stat() results in the file manager would
trigger a second rebuild of the already-up-to-date module, causing
failures down the line.

The global module index now lazily resolves its module file references
to actual AST reader module files only after the module file has been
loaded, eliminating the stat-caching race. Moreover, the AST reader
can communicate to its caller that a module file is missing (rather
than simply being out-of-date), allowing us to simplify the
module-loading logic and allowing the compiler to recover if a
dependent module file ends up getting deleted.

llvm-svn: 177367

show more ...

Never cache the result of a module file lookup.

llvm-svn: 174744