[llvm-cov] Support directory layout in coverage reports

This is a GSoC 2023 project ([discourse link](https://discourse.llvm.org/t/coverage-support-a-hierarchical-directory-structure-in-generated-co

[llvm-cov] Support directory layout in coverage reports

This is a GSoC 2023 project ([discourse link](https://discourse.llvm.org/t/coverage-support-a-hierarchical-directory-structure-in-generated-coverage-html-reports/68239)).

llvm-cov currently generates a single top-level index HTML file, which causes rendering scalability issues in large projects. This patch adds support for hierarchical directory structure into the HTML reports to solve scalability issues by introducing the following changes:

- Added a new command line option `--show-directory-coverage` for `llvm-cov show`. It works both for `--format=html` and `--format=text`.
- Two new classes: `CoveragePrinterHTMLDirectory` and `CoveragePrinterTextDirectory` was added to support the new option.
- A tool class `DirectoryCoverageReport` was added to support the two classes above.
- Updated the document.
- Added a new regression test for `--show-directory-coverage`.

Reviewed By: phosek, gulfem

Differential Revision: https://reviews.llvm.org/D151283

show more ...

Reland "[llvm-cov] Support multi-source object files for convert-for-testing"

`llvm-cov convert-for-testing` only functions properly when the input binary contains a single source file. When the bin

Reland "[llvm-cov] Support multi-source object files for convert-for-testing"

`llvm-cov convert-for-testing` only functions properly when the input binary contains a single source file. When the binary has multiple source files, a `Malformed coverage data` error will occur when the generated .covmapping is read back. This is because the testing format lacks support for indicating the size of its file records, and current implementation just assumes there's only one record in it. This patch fixes this problem by introducing a new testing format version.

Changes to the code:

- Add a new format version. The version number is stored in the the last 8 bytes of the orignial magic number field to be backward-compatible.
- Output a LEB128 number before the file records section to indicate its size in the new version.
- Change the format parsing code correspondingly.
- Update the document to formalize the testing format.
- Additionally, fix the bug when converting COFF binaries.

Reviewed By: phosek, gulfem

Differential Revision: https://reviews.llvm.org/D156611

show more ...

Revert "[llvm-cov] Fix a bug about using `convert-for-testing` on multi-source object files"

This reverts commit f8ad86c23405168a8cd189590184fdcc296627e0 because
it broke some downsteam tests report

Revert "[llvm-cov] Fix a bug about using `convert-for-testing` on multi-source object files"

This reverts commit f8ad86c23405168a8cd189590184fdcc296627e0 because
it broke some downsteam tests reported in https://reviews.llvm.org/D156611.

show more ...

[llvm-cov] Fix a bug about using `convert-for-testing` on multi-source object files

`llvm-cov convert-for-testing` is used to build the .covmapping files used in its regression tests. However the cu

[llvm-cov] Fix a bug about using `convert-for-testing` on multi-source object files

`llvm-cov convert-for-testing` is used to build the .covmapping files used in its regression tests. However the current implementation only works when there's only one source file in the mapping information data. If there are more than 1 source files, `llvm-cov convert-for-testing` can still produce a .covmapping file, but when read it back, `llvm-cov` will report:

```
error: Failed to load coverage: 'main.covmapping': Malformed coverage data
```

This is because the output .covmapping file doesn't have any mark to indicate the boundary between file records and function records, and current implementation jsut assume there's only one file record in the .covmapping file.

Changes to the code:

- Make `llvm-cov convert-for-testing` output a LEB128 number before file records to indicate its size.
- Change the testing format parsing code correspondingly.
- Update existing .covmapping files.

Reviewed By: gulfem

Differential Revision: https://reviews.llvm.org/D156611

show more ...

Cleanup LLVMObject headers

Most notably,

llvm/Object/Binary.h no longer includes llvm/Support/MemoryBuffer.h
llvm/Object/MachOUniversal*.h no longer include llvm/Object/Archive.h
llvm/Object/TapiUn

Cleanup LLVMObject headers

Most notably,

llvm/Object/Binary.h no longer includes llvm/Support/MemoryBuffer.h
llvm/Object/MachOUniversal*.h no longer include llvm/Object/Archive.h
llvm/Object/TapiUniversal.h no longer includes llvm/Object/TapiFile.h

llvm-project preprocessed size:
before: 1068185081
after: 1068324320

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D119457

show more ...

[llvm-cov] Support for v4 format in convert-for-testing

v4 moves function records to a dedicated section so we need to write
and read it separately.

https://reviews.llvm.org/D100535

[Support] Don't include VirtualFileSystem.h in CommandLine.h

CommandLine.h is indirectly included in ~50% of TUs when building
clang, and VirtualFileSystem.h is large.

(Already remarked by jhenders

[Support] Don't include VirtualFileSystem.h in CommandLine.h

CommandLine.h is indirectly included in ~50% of TUs when building
clang, and VirtualFileSystem.h is large.

(Already remarked by jhenderson on D70769.)

No behavior change.

Differential Revision: https://reviews.llvm.org/D100957

show more ...

[Alignment][NFC] Remove unneeded llvm:: scoping on Align types

llvm-svn: 373081

[Alignment] Move OffsetToAlignment to Alignment.h

Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/201

[Alignment] Move OffsetToAlignment to Alignment.h

Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Reviewers: courbet, JDevlieghere, alexshap, rupprecht, jhenderson

Subscribers: sdardis, nemanjai, hiraditya, kbarton, jakehehrlich, jrtc27, MaskRay, atanasyan, jsji, seiya, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D67499

llvm-svn: 371742

show more ...

Recommit r368812 "[llvm/Object] - Convert SectionRef::getName() to return Expected<>"

Changes: no changes. A fix for the clang code will be landed right on top.

Original commit message:

SectionRef

Recommit r368812 "[llvm/Object] - Convert SectionRef::getName() to return Expected<>"

Changes: no changes. A fix for the clang code will be landed right on top.

Original commit message:

SectionRef::getName() returns std::error_code now.
Returning Expected<> instead has multiple benefits.

For example, it forces user to check the error returned.
Also Expected<> may keep a valuable string error message,
what is more useful than having a error code.
(Object\invalid.test was updated to show the new messages printed.)

This patch makes a change for all users to switch to Expected<> version.

Note: in a few places the error returned was ignored before my changes.
In such places I left them ignored. My intention was to convert the interface
used, and not to improve and/or the existent users in this patch.
(Though I think this is good idea for a follow-ups to revisit such places
and either remove consumeError calls or comment each of them to clarify why
it is OK to have them).

Differential revision: https://reviews.llvm.org/D66089

llvm-svn: 368826

show more ...

Revert r368812 "[llvm/Object] - Convert SectionRef::getName() to return Expected<>"

It broke clang BB: http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/16455

llvm-svn: 368813

[llvm/Object] - Convert SectionRef::getName() to return Expected<>

SectionRef::getName() returns std::error_code now.
Returning Expected<> instead has multiple benefits.

For example, it forces user

[llvm/Object] - Convert SectionRef::getName() to return Expected<>

SectionRef::getName() returns std::error_code now.
Returning Expected<> instead has multiple benefits.

For example, it forces user to check the error returned.
Also Expected<> may keep a valuable string error message,
what is more useful than having a error code.
(Object\invalid.test was updated to show the new messages printed.)

This patch makes a change for all users to switch to Expected<> version.

Note: in a few places the error returned was ignored before my changes.
In such places I left them ignored. My intention was to convert the interface
used, and not to improve and/or the existent users in this patch.
(Though I think this is good idea for a follow-ups to revisit such places
and either remove consumeError calls or comment each of them to clarify why
it is OK to have them).

Differential revision: https://reviews.llvm.org/D66089

llvm-svn: 368812

show more ...

Recommit [Object] Change object::SectionRef::getContents() to return Expected<StringRef>

r360876 didn't fix 2 call sites in clang.

Expected<ArrayRef<uint8_t>> may be better but use Expected<StringR

Recommit [Object] Change object::SectionRef::getContents() to return Expected<StringRef>

r360876 didn't fix 2 call sites in clang.

Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.

Follow-up of D61781.

llvm-svn: 360892

show more ...

Revert r360876 "[Object] Change object::SectionRef::getContents() to return Expected<StringRef>"

It broke the Clang build, see llvm-commits thread.

> Expected<ArrayRef<uint8_t>> may be better but u

Revert r360876 "[Object] Change object::SectionRef::getContents() to return Expected<StringRef>"

It broke the Clang build, see llvm-commits thread.

> Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.
>
> Follow-up of D61781.

llvm-svn: 360878

show more ...

[Object] Change object::SectionRef::getContents() to return Expected<StringRef>

Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.

Follow-up of D61781.

llvm-svn: 360876

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[Support] Make error banner optional in logAllUnhandledErrors

In a lot of places an empty string was passed as the ErrorBanner to
logAllUnhandledErrors. This patch makes that argument optional to
si

[Support] Make error banner optional in logAllUnhandledErrors

In a lot of places an empty string was passed as the ErrorBanner to
logAllUnhandledErrors. This patch makes that argument optional to
simplify the call sites.

llvm-svn: 346604

show more ...

[FileSystem] Split up the OpenFlags enumeration.

This breaks the OpenFlags enumeration into two separate
enumerations: OpenFlags and CreationDisposition. The first
controls the behavior of the API

[FileSystem] Split up the OpenFlags enumeration.

This breaks the OpenFlags enumeration into two separate
enumerations: OpenFlags and CreationDisposition. The first
controls the behavior of the API depending on whether or not
the target file already exists, and is not a flags-based
enum. The second controls more flags-like values.

This yields a more easy to understand API, while also allowing
flags to be passed to the openForRead api, where most of the
values didn't make sense before. This also makes the apis more
testable as it becomes easy to enumerate all the configurations
which make sense, so I've added many new tests to exercise all
the different values.

llvm-svn: 334221

show more ...

[ProfileData] Unify getInstrProf*SectionName helpers

This is a version of D32090 that unifies all of the
`getInstrProf*SectionName` helper functions. (Note: the build failures
which D32090 would hav

[ProfileData] Unify getInstrProf*SectionName helpers

This is a version of D32090 that unifies all of the
`getInstrProf*SectionName` helper functions. (Note: the build failures
which D32090 would have addressed were fixed with r300352.)

We should unify these helper functions because they are hard to use in
their current form. E.g we recently introduced more helpers to fix
section naming for COFF files. This scheme doesn't totally succeed at
hiding low-level details about section naming, so we should switch to an
API that is easier to maintain.

This is not an NFC commit because it fixes llvm-cov's testing support
for COFF files (this falls out of the API change naturally). This is an
area where we lack tests -- I will see about adding one as a follow up.

Testing: check-clang, check-profile, check-llvm.

Differential Revision: https://reviews.llvm.org/D32097

llvm-svn: 300381

show more ...

[Profile] PE binary coverage bug fix

PR/32584

Differential Revision: https://reviews.llvm.org/D32023

llvm-svn: 300277

Search for llvm-symbolizer binary in the same directory as argv[0], before
looking for it along $PATH. This allows installs of LLVM tools outside of
$PATH to find the symbolizer and produce pretty ba

Search for llvm-symbolizer binary in the same directory as argv[0], before
looking for it along $PATH. This allows installs of LLVM tools outside of
$PATH to find the symbolizer and produce pretty backtraces if they crash.

llvm-svn: 272232

show more ...

[llvm-cov] Use string getters (NFC)

llvm-svn: 271537

[Coverage] Ensure that coverage mapping data has an expected alignment in 'covmapping' files.

Coverage mapping data is organized in a sequence of blocks, each of which is expected
to be aligned by 8

[Coverage] Ensure that coverage mapping data has an expected alignment in 'covmapping' files.

Coverage mapping data is organized in a sequence of blocks, each of which is expected
to be aligned by 8 bytes. This feature is used when reading those blocks, see
VersionedCovMapFuncRecordReader::readFunctionRecords(). If a misaligned covearge
mapping data has more than one block, it causes llvm-cov to fail.

Differential Revision: http://reviews.llvm.org/D20285

llvm-svn: 269887

show more ...

Thread Expected<...> up from createMachOObjectFile() to allow llvm-objdump to produce a real error message

Produce the first specific error message for a malformed Mach-O file describing
the problem

Thread Expected<...> up from createMachOObjectFile() to allow llvm-objdump to produce a real error message

Produce the first specific error message for a malformed Mach-O file describing
the problem instead of the generic message for object_error::parse_failed of
"Invalid data was encountered while parsing the file”.  Many more good error
messages will follow after this first one.

This is built on Lang Hames’ great work of adding the ’Error' class for
structured error handling and threading Error through MachOObjectFile
construction. And making createMachOObjectFile return Expected<...> .

So to to get the error to the llvm-obdump tool, I changed the stack of
these methods to also return Expected<...> :

object::ObjectFile::createObjectFile()
object::SymbolicFile::createSymbolicFile()
object::createBinary()

Then finally in ParseInputMachO() in MachODump.cpp the error can
be reported and the specific error message can be printed in llvm-objdump
and can be seen in the existing test case for the existing malformed binary
but with the updated error message.

Converting these interfaces to Expected<> from ErrorOr<> does involve
touching a number of places. To contain the changes for now use of
errorToErrorCode() and errorOrToExpected() are used where the callers
are yet to be converted.

Also there some were bugs in the existing code that did not deal with the
old ErrorOr<> return values. So now with Expected<> since they must be
checked and the error handled, I added a TODO and a comment:
“// TODO: Actually report errors helpfully” and a call something like
consumeError(ObjOrErr.takeError()) so the buggy code will not crash
since needed to deal with the Error.

Note there is one fix also needed to lld/COFF/InputFiles.cpp that goes along
with this that I will commit right after this. So expect lld not to built
after this commit and before the next one.

llvm-svn: 265606

show more ...

[cleanup] Re-sort all the #include lines in LLVM using
utils/sort_includes.py.

I clearly haven't done this in a while, so more changed than usual. This
even uncovered a missing include from the Inst

[cleanup] Re-sort all the #include lines in LLVM using
utils/sort_includes.py.

I clearly haven't done this in a while, so more changed than usual. This
even uncovered a missing include from the InstrProf library that I've
added. No functionality changed here, just mechanical cleanup of the
include order.

llvm-svn: 225974

show more ...