Revert "[VFS] Add 'overlay-relative' field to YAML files" and "[VFS] Fix call to getVFSFromYAML in unittests"

This reverts commit r261552 and r261556 because of failing unittests on
windows:

Failin

Revert "[VFS] Add 'overlay-relative' field to YAML files" and "[VFS] Fix call to getVFSFromYAML in unittests"

This reverts commit r261552 and r261556 because of failing unittests on
windows:

Failing Tests (4):
Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.CaseInsensitive
Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.DirectoryIteration
Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.MappedFiles
Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.UseExternalName

llvm-svn: 261613

show more ...

[VFS] Add 'overlay-relative' field to YAML files

The VFS overlay mapping between virtual paths and real paths is done through
the 'external-contents' entries in YAML files, which contains hardcoded

[VFS] Add 'overlay-relative' field to YAML files

The VFS overlay mapping between virtual paths and real paths is done through
the 'external-contents' entries in YAML files, which contains hardcoded paths
to the real files.

When a module compilation crashes, headers are dumped into <name>.cache/vfs
directory and are mapped via the <name>.cache/vfs/vfs.yaml. The script
generated for reproduction uses -ivfsoverlay pointing to file to gather the
mapping between virtual paths and files inside <name>.cache/vfs. Currently, we
are only capable of reproducing such crashes in the same machine as they
happen, because of the hardcoded paths in 'external-contents'.

To be able to reproduce a crash in another machine, this patch introduces a new
option in the VFS yaml file called 'overlay-relative'. When it's equal to
'true' it means that the provided path to the YAML file through the
-ivfsoverlay option should also be used to prefix the final path for every
'external-contents'.

Example, given the invocation snippet "... -ivfsoverlay
<name>.cache/vfs/vfs.yaml" and the following entry in the yaml file:

"overlay-relative": "true",
"roots": [
...
"type": "directory",
"name": "/usr/include",
"contents": [
{
"type": "file",
"name": "stdio.h",
"external-contents": "/usr/include/stdio.h"
},
...

Here, a file manager request for virtual "/usr/include/stdio.h", that will map
into real path "/<absolute_path_to>/<name>.cache/vfs/usr/include/stdio.h.

This is a useful feature for debugging module crashes in machines other than
the one where the error happened.

Differential Revision: http://reviews.llvm.org/D17457

rdar://problem/24499339

llvm-svn: 261552

show more ...

[VFS] Add support for handling path traversals

Handle ".", ".." and "./" with trailing slashes while collecting files
to be dumped into the vfs overlay directory.

Include the support for symlinks i

[VFS] Add support for handling path traversals

Handle ".", ".." and "./" with trailing slashes while collecting files
to be dumped into the vfs overlay directory.

Include the support for symlinks into components. Given the path:

/install-dir/bin/../lib/clang/3.8.0/include/altivec.h, if "bin"
component is a symlink, it's not safe to use `path::remove_dots` here,
and `realpath` is used to get the right answer. Since `realpath`
is expensive, we only do it at collecting time (which only happens
during the crash reproducer) and cache the base directory for fast lookups.

Overall, this makes the input to the VFS YAML file to be canonicalized
to never contain traversal components.

Differential Revision: http://reviews.llvm.org/D17104

rdar://problem/24499339

llvm-svn: 261551

show more ...

Moving FileManager::removeDotPaths to llvm::sys::path::remove_dots

Differential Revision: http://reviews.llvm.org/D14394

llvm-svn: 252501

[modules] Change the way we deal with .d output for explicitly-specified module
files: include the .pcm file itself in the .d output, rather than including its
own input files. Other forms of module

[modules] Change the way we deal with .d output for explicitly-specified module
files: include the .pcm file itself in the .d output, rather than including its
own input files. Other forms of module file continue to be transparent for .d
output.

Arguably, the input files for the .pcm file are still inputs to the
compilation, but that's unnecessary for make-like build systems (where the
mtime of the .pcm file is sufficient) and harmful for smarter build systems
that know about module files and want to track only the local dependencies.

llvm-svn: 244923

show more ...

Revert r240270 ("Fixed/added namespace ending comments using clang-tidy").

llvm-svn: 240353

Fixed/added namespace ending comments using clang-tidy. NFC

The patch is generated using this command:

$ tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comme

Fixed/added namespace ending comments using clang-tidy. NFC

The patch is generated using this command:

$ tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
work/llvm/tools/clang

To reduce churn, not touching namespaces spanning less than 10 lines.

llvm-svn: 240270

show more ...

Remove many superfluous SmallString::str() calls.

Now that SmallString is a first-class citizen, most SmallString::str()
calls are not required. This patch removes a whole bunch of them, yet
there a

Remove many superfluous SmallString::str() calls.

Now that SmallString is a first-class citizen, most SmallString::str()
calls are not required. This patch removes a whole bunch of them, yet
there are lots more.

There are two use cases where str() is really needed:
1) To use one of StringRef member functions which is not available in
SmallString.
2) To convert to std::string, as StringRef implicitly converts while
SmallString do not. We may wish to change this, but it may introduce
ambiguity.

llvm-svn: 232622

show more ...

[cleanup] Re-sort *all* #include lines with llvm/utils/sort_includes.py

Sorry for the noise, I managed to miss a bunch of recent regressions of
include orderings here. This should actually sort all

[cleanup] Re-sort *all* #include lines with llvm/utils/sort_includes.py

Sorry for the noise, I managed to miss a bunch of recent regressions of
include orderings here. This should actually sort all the includes for
Clang. Again, no functionality changed, this is just a mechanical
cleanup that I try to run periodically to keep the #include lines as
regular as possible across the project.

llvm-svn: 225979

show more ...

Frontend: Canonicalize to native paths when dumping module dependencies

Mixed path separators (ie, both / and \\) can mess up the sort order
of the VFS map when dumping module dependencies, as was r

Frontend: Canonicalize to native paths when dumping module dependencies

Mixed path separators (ie, both / and \\) can mess up the sort order
of the VFS map when dumping module dependencies, as was recently
exposed by r224055 and papered over in r224145. Instead, we should
simply use native paths for consistency.

This also adds a TODO to add handling of .. in paths. There was some
code for this before r224055, but it was untested and probably broken.

llvm-svn: 224164

show more ...

[modules] When constructing paths relative to a module, strip out /./ directory
components. These sometimes get synthetically added, and we don't want -Ifoo
and -I./foo to be treated fundamentally di

[modules] When constructing paths relative to a module, strip out /./ directory
components. These sometimes get synthetically added, and we don't want -Ifoo
and -I./foo to be treated fundamentally differently here.

llvm-svn: 224055

show more ...

Update for llvm api change.

llvm-svn: 216397

unique_ptr-ify ASTReaderListener in the ASTReader

llvm-svn: 215317

Frontend: Disentangle removePathTraversal from concatenating paths

This reimplements part of r211303 in a bit of a cleaner way. Doing so
allows us to use a proper absolute path when calling addFileM

Frontend: Disentangle removePathTraversal from concatenating paths

This reimplements part of r211303 in a bit of a cleaner way. Doing so
allows us to use a proper absolute path when calling addFileMapping
rather than relying on a substring being one, which should fix the
tests on Windows.

llvm-svn: 211338

show more ...

Frontend: Fix a typo

llvm-svn: 211306

Frontend: Add a CC1 flag to dump module dependencies to a directory

This adds the -module-dependency-dir to clang -cc1, which specifies a
directory to copy all of a module's dependencies into in a f

Frontend: Add a CC1 flag to dump module dependencies to a directory

This adds the -module-dependency-dir to clang -cc1, which specifies a
directory to copy all of a module's dependencies into in a form
suitable to be used as a VFS using -ivfsoverlay with the generated
vfs.yaml.

This is useful for crashdumps that involve modules, so that the module
dependencies will be intact when a crash report script is used to
reproduce a problem on another machine.

We currently encode the absolute path to the dump directory, due to
limitations in the VFS system. Until we can handle relative paths in
the VFS, users of the VFS map may need to run a simple search and
replace in the file.

llvm-svn: 211303

show more ...