[ThinLTO][NFC] Refactor FileCache (#110463)

This is a prep for https://github.com/llvm/llvm-project/pull/90933.

- Change `FileCache` from a function to a type.
- Store the cache directory i

[ThinLTO][NFC] Refactor FileCache (#110463)

This is a prep for https://github.com/llvm/llvm-project/pull/90933.

- Change `FileCache` from a function to a type.
- Store the cache directory in the type, which will be used when creating additional caches for two-codegen round runs that inherit this value.

show more ...

[Support] Use SmallString::operator std::string (NFC)

[Support] Better error msg when cache dir can't be created. (#69575)

On windows if you passed /lldltocache:D:\tmp to lld and you didn't have
D: mounted it fail to create the cache dir D:\tmp, but t

[Support] Better error msg when cache dir can't be created. (#69575)

On windows if you passed /lldltocache:D:\tmp to lld and you didn't have
D: mounted it fail to create the cache dir D:\tmp, but the error message
is pretty hard to understand:

```
c:\code\llvm\llvm-project\out\debug>bin\lld-link.exe /lldltocache:D:\tmp
hello.obj
LLVM ERROR: no such file or directory

PLEASE submit a bug report to
https://github.com/llvm/llvm-project/issues/ and include the crash
backtrace.
Exception Code: 0xC000001D
```

Which lead one of our users to report this as a crash. I have just added
a bit better message so it now says:

```
c:\code\llvm\llvm-project\out\debug>bin\lld-link.exe /lldltocache:D:\tmp
hello.obj
LLVM ERROR: Can't create cache directory: D:\tmp

PLEASE submit a bug report to
https://github.com/llvm/llvm-project/issues/ and include the crash
backtrace.
```

I am not sure this is a fatal error because it's not something that
really should be reported as a bug to LLVM. But at least this gives a
bit more visibility on what to change.

show more ...

Reland "[LTO][COFF] Use bitcode file names in lto native object file names."

This reverts commit 34108082947c964ae9bbfcd9808f2fd31c0d672f with fixes.

Revert "Reland "[LTO][COFF] Use bitcode file names in lto native object file names.""

Breaks build of LLVMgold here:
```
/repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1108:19: error: no

Revert "Reland "[LTO][COFF] Use bitcode file names in lto native object file names.""

Breaks build of LLVMgold here:
```
/repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1108:19: error: no matching function for call to 'localCache'
Cache = check(localCache("ThinLTO", "Thin", options::cache_dir, AddBuffer));
^~~~~~~~~~
/repositories/llvm-project/llvm/include/llvm/Support/Caching.h:72:21: note: candidate function not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1102:20)' to 'llvm::AddBufferFn' (aka 'function<void (unsigned int, const llvm::Twine &, std::unique_ptr<MemoryBuffer>)>') for 4th argument
Expected<FileCache> localCache(
^
/repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1110:18: error: no viable conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'llvm::AddStreamFn' (aka 'function<Expected<std::unique_ptr<CachedFileStream>> (unsigned int, const llvm::Twine &)>')
check(Lto->run(AddStream, Cache));
^~~~~~~~~
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:375:7: note: candidate constructor not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'std::nullptr_t' for 1st argument
function(nullptr_t) noexcept
^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:386:7: note: candidate constructor not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'const std::function<llvm::Expected<std::unique_ptr<llvm::CachedFileStream>> (unsigned int, const llvm::Twine &)> &' for 1st argument
function(const function& __x)
^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:404:7: note: candidate constructor not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'std::function<llvm::Expected<std::unique_ptr<llvm::CachedFileStream>> (unsigned int, const llvm::Twine &)> &&' for 1st argument
function(function&& __x) noexcept
^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:435:2: note: candidate template ignored: requirement '_Callable<(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20) &, (lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20), std::__invoke_result<(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20) &, unsigned int, const llvm::Twine &>>::value' was not satisfied [with _Functor = (lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20) &]
function(_Functor&& __f)
^
/repositories/llvm-project/llvm/include/llvm/LTO/LTO.h:278:25: note: passing argument to parameter 'AddStream' here
Error run(AddStreamFn AddStream, FileCache Cache = nullptr);
^

```

This reverts commit 387620aa8cea33174b6c1fb80c1af713fee732ac.

show more ...

Reland "[LTO][COFF] Use bitcode file names in lto native object file names."

This reverts commit eef5405f74ae208e3e2eb7daacecac923d7338f2.

Revert "[LTO][COFF] Use bitcode file names in lto native object file names."

This reverts commit 531ed6d5aa65f41c6dfe2e74905d5c6c88fc95a7.

[LTO][COFF] Use bitcode file names in lto native object file names.

Currently the lto native object files have names like main.exe.lto.1.obj. In
PDB, those names are used as names for each compiland

[LTO][COFF] Use bitcode file names in lto native object file names.

Currently the lto native object files have names like main.exe.lto.1.obj. In
PDB, those names are used as names for each compiland. Microsoft’s tool
SizeBench uses those names to present to users the size of each object files.
So, names like main.exe.lto.1.obj is not user friendly.

This patch makes the lto native object file names more readable by using
the bitcode file names as part of the file names. For example, if the input
bitcode file has path like "path/to/foo.obj", its corresponding lto native
object file path would be "path/to/main.exe.lto.foo.obj". Since the lto native
object file name only bothers PDB, this patch only changes the lld-linker's
behavior.

Reviewed By: tejohnson, MaskRay, #lld-macho

Differential Revision: https://reviews.llvm.org/D137217

show more ...

[Support] [DebugInfo] Lazily create cache dir.

This change defers creating Support/Caching.cpp's cache directory until
it actually writes to the cache.

This allows using Caching library in a read-o

[Support] [DebugInfo] Lazily create cache dir.

This change defers creating Support/Caching.cpp's cache directory until
it actually writes to the cache.

This allows using Caching library in a read-only fashion. If read-only,
the cache is guaranteed not to write to disk. This keeps tools using
DebugInfod (currently llvm-symbolizer) hermetic when not configured to
perform remote lookups.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D117589

show more ...

Reland - [CodeView] Emit S_OBJNAME record

Reland integrates build fixes & further review suggestions.

Thanks to @zturner for the initial S_OBJNAME patch!

Differential Revision: https://reviews.llv

Reland - [CodeView] Emit S_OBJNAME record

Reland integrates build fixes & further review suggestions.

Thanks to @zturner for the initial S_OBJNAME patch!

Differential Revision: https://reviews.llvm.org/D43002

show more ...

Revert [CodeView] Emit S_OBJNAME record

Also revert all subsequent fixes:
- abd1cbf5e543f0f114d2742e109ead7d7ddbf9c4 [Clang] Disable debug-info-objname.cpp test on Unix until I sort out the issue.
-

Revert [CodeView] Emit S_OBJNAME record

Also revert all subsequent fixes:
- abd1cbf5e543f0f114d2742e109ead7d7ddbf9c4 [Clang] Disable debug-info-objname.cpp test on Unix until I sort out the issue.
- 00ec441253048f5e30540ea26bb0a28c42a5fc18 [Clang] debug-info-objname.cpp test: explictly encode a x86 target when using %clang_cl to avoid falling back to a native CPU triple.
- cd407f6e52b09cce2bef24c74b7f36fedc94991b [Clang] Fix build by restricting debug-info-objname.cpp test to x86.

show more ...

[CodeView] Emit S_OBJNAME record

Thanks to @zturner for the initial patch!

Differential Revision: https://reviews.llvm.org/D43002

[Support] Use final filename for Caching buffer identifier

Mach-O LLD uses the buffer identifier of the memory buffer backing an object
file to generate stabs which are used by `dsymutil` to find th

[Support] Use final filename for Caching buffer identifier

Mach-O LLD uses the buffer identifier of the memory buffer backing an object
file to generate stabs which are used by `dsymutil` to find the object file for
dSYM generation.

When using thinLTO, these buffers are provided by the cache which initially
saves them to disk as temporary files beginning with "Thin-" but renames them
to persistent files beginning with "llvmcache-" before the buffer is provided
to the cache user.

However, the buffer is created before the file is renamed and is given the temp
file's name as an identifier. This causes the generated stabs to point to
nonexistent files.

This change names the buffer with the eventual persistent filename. I think
this is safe because failing to rename the temp file is a fatal error.

Differential Revision: https://reviews.llvm.org/D115055

show more ...

[Support] Improve Caching conformance with Support library behavior

This diff makes several amendments to the local file caching mechanism
which was migrated from ThinLTO to Support in
rGe678c511771

[Support] Improve Caching conformance with Support library behavior

This diff makes several amendments to the local file caching mechanism
which was migrated from ThinLTO to Support in
rGe678c51177102845c93529d457b020f969125373 in response to follow-up
discussion on that commit.

Patch By: noajshu

Differential Revision: https://reviews.llvm.org/D113080

show more ...

[Support][ThinLTO] Move ThinLTO caching to LLVM Support library

We would like to move ThinLTO’s battle-tested file caching mechanism to
the LLVM Support library so that we can use it elsewhere in LL

[Support][ThinLTO] Move ThinLTO caching to LLVM Support library

We would like to move ThinLTO’s battle-tested file caching mechanism to
the LLVM Support library so that we can use it elsewhere in LLVM.

Patch By: noajshu

Differential Revision: https://reviews.llvm.org/D111371

show more ...

[Support][ThinLTO] Move ThinLTO caching to LLVM Support library

We would like to move ThinLTO’s battle-tested file caching mechanism to
the LLVM Support library so that we can use it elsewhere in LL

[Support][ThinLTO] Move ThinLTO caching to LLVM Support library

We would like to move ThinLTO’s battle-tested file caching mechanism to
the LLVM Support library so that we can use it elsewhere in LLVM.

Patch By: noajshu

Differential Revision: https://reviews.llvm.org/D111371

show more ...