[SystemZ][z/OS] Open YAML files for read as text files (#121340)

This patch makes sure YAML files are opened for reading as text file to
trigger auto-conversion from EBCDIC encoding into expected A

[SystemZ][z/OS] Open YAML files for read as text files (#121340)

This patch makes sure YAML files are opened for reading as text file to
trigger auto-conversion from EBCDIC encoding into expected ASCII
encoding on z/OS platform. This is required to fix the following lit
tests:

```
LLVM :: tools/llvm-gsymutil/ARM_AArch64/macho-gsym-callsite-info-exe.yaml
LLVM :: tools/llvm-gsymutil/ARM_AArch64/macho-gsym-callsite-info-obj.test
LLVM :: tools/llvm-gsymutil/ARM_AArch64/macho-gsym-callsite-info-dsym.yaml
LLVM :: Transforms/PGOProfile/memprof_undrift_missing_leaf.ll
```

show more ...

[memprof] clang-format MemProf-related files (NFC) (#120504)

[memprof] Move YAML support to MemProfYAML.h (NFC) (#119515)

The YAML support is increasing in size, so this patch moves it to a
separate file.

[memprof] Add YAML read/write support to llvm-profdata (#118915)

This patch adds YAML read/write support to llvm-profdata. The primary
intent is to accommodate MemProf profiles in test cases, ther

[memprof] Add YAML read/write support to llvm-profdata (#118915)

This patch adds YAML read/write support to llvm-profdata. The primary
intent is to accommodate MemProf profiles in test cases, thereby
avoiding the binary format.

The read support is via llvm-profdata merge. This is useful when we
want to verify that the compiler does the right thing on a given .ll
file and a MemProf profile in a test case. In the test case, we would
convert the MemProf profile in YAML to an indexed profile and invoke
the compiler on the .ll file along with the indexed profile.

The write support is via llvm-profdata show --memory. This is useful
when we wish to convert an indexed MemProf profile to YAML while
writing tests. We would compile a test case in C++, run it for an
indexed MemProf profile, and then convert it to the text format.

show more ...

[ProfileData] Avoid deprecated is_pod

[memprof] Add IndexedMemProfData::addCallStack (#118920)

This patch adds a helper function to replace an idiom like:

CallStackId CSId = hashCallStack(CallStack)
MemProfData.CallStacks.try_em

[memprof] Add IndexedMemProfData::addCallStack (#118920)

This patch adds a helper function to replace an idiom like:

CallStackId CSId = hashCallStack(CallStack)
MemProfData.CallStacks.try_emplace(CSId, CallStack);
// Do something with CSId.

show more ...

[memprof] Add IndexedMemProfData::addFrame (#118724)

This patch adds a helper function to replace an idiom like:

FrameId Id = F.hash();
MemProfData.Frames.try_emplace(Id, F);
// Do someth

[memprof] Add IndexedMemProfData::addFrame (#118724)

This patch adds a helper function to replace an idiom like:

FrameId Id = F.hash();
MemProfData.Frames.try_emplace(Id, F);
// Do something with Id.

show more ...

[memprof] Move YAML traits to MemProf.h (NFC) (#118668)

This patch moves the MemProf YAML traits to MemProf.h so that the YAML
writer can access them from outside MemProfReader.cpp in the future.

[memprof] Remove inline call stacks (#117833)

Now that MemProf format version 1 has been removed, nobody uses:

- IndexedAllocationInfo::CallStack
- IndexedMemProfRecord::CallSites

This patch

[memprof] Remove inline call stacks (#117833)

Now that MemProf format version 1 has been removed, nobody uses:

- IndexedAllocationInfo::CallStack
- IndexedMemProfRecord::CallSites

This patch removed the dead struct fields.

You might notice that IndexedMemProfRecord::{clear,merge} do not
mention CallSiteIds at all. I think it's an oversight. clear doesn't
matter at the moment because we call it during serialization to reduce
memory footprint. merge is simply not as well tested as it should be.
I'll follow up with a separate patch to address these issues.

show more ...

[memprof] Fix warnings on MSVC

MSVC doesn't seem to count a use in static_assert as a use.

Reapply [memprof] Add YAML-based deserialization for MemProf profile (#117829)

This patch adds YAML-based deserialization for MemProf profile.

It's been painful to write tests for MemProf passes be

Reapply [memprof] Add YAML-based deserialization for MemProf profile (#117829)

This patch adds YAML-based deserialization for MemProf profile.

It's been painful to write tests for MemProf passes because we do not
have a text format for the MemProf profile. We would write a test
case in C++, run it for a binary MemProf profile, and then finally run
a test written in LLVM IR with the binary profile.

This patch paves the way toward YAML-based MemProf profile.
Specifically, it adds new class YAMLMemProfReader derived from
MemProfReader. For now, it only adds a function to parse StringRef
pointing to YAML data. Subseqeunt patches will wire it to
llvm-profdata and read from a file.

The field names are based on various printYAML functions in MemProf.h.
I'm not aiming for compatibility with the format used in printYAML,
but I don't see a point in changing the field names.

This iteration works around the unavailability of
ScalarTraits<uintptr_t> on macOS.

show more ...

Revert "[memprof] Add YAML-based deserialization for MemProf profile (#117829)"

This reverts commit c00e53208db638c35499fc80b555f8e14baa35f0.

It looks like this breaks building LLVM on macOS and so

Revert "[memprof] Add YAML-based deserialization for MemProf profile (#117829)"

This reverts commit c00e53208db638c35499fc80b555f8e14baa35f0.

It looks like this breaks building LLVM on macOS and some other
platform/compiler combos

https://lab.llvm.org/buildbot/#/builders/23/builds/5252
https://green.lab.llvm.org/job/llvm.org/job/clang-san-iossim/5356/console

In file included from /Users/ec2-user/jenkins/workspace/llvm.org/clang-san-iossim/llvm-project/llvm/lib/ProfileData/MemProfReader.cpp:34:
In file included from /Users/ec2-user/jenkins/workspace/llvm.org/clang-san-iossim/llvm-project/llvm/include/llvm/ProfileData/MemProfReader.h:24:
In file included from /Users/ec2-user/jenkins/workspace/llvm.org/clang-san-iossim/llvm-project/llvm/include/llvm/ProfileData/InstrProfReader.h:22:
In file included from /Users/ec2-user/jenkins/workspace/llvm.org/clang-san-iossim/llvm-project/llvm/include/llvm/ProfileData/InstrProfCorrelator.h:21:
/Users/ec2-user/jenkins/workspace/llvm.org/clang-san-iossim/llvm-project/llvm/include/llvm/Support/YAMLTraits.h:1173:36: error: implicit instantiation of undefined template 'llvm::yaml::MissingTrait<unsigned long>'
char missing_yaml_trait_for_type[sizeof(MissingTrait<T>)];
^
/Users/ec2-user/jenkins/workspace/llvm.org/clang-san-iossim/llvm-project/llvm/include/llvm/Support/YAMLTraits.h:961:7: note: in instantiation of function template specialization 'llvm::yaml::yamlize<unsigned long>' requested here
yamlize(*this, Val, Required, Ctx);
^
/Users/ec2-user/jenkins/workspace/llvm.org/clang-san-iossim/llvm-project/llvm/include/llvm/Support/YAMLTraits.h:883:11: note: in instantiation of function template specialization 'llvm::yaml::IO::processKey<unsigned long, llvm::yaml::EmptyContext>' requested here
this->processKey(Key, Val, true, Ctx);
^
/Users/ec2-user/jenkins/workspace/llvm.org/clang-san-iossim/llvm-project/llvm/include/llvm/ProfileData/MIBEntryDef.inc:55:1: note: in instantiation of function template specialization 'llvm::yaml::IO::mapRequired<unsigned long>' requested here
MIBEntryDef(AccessHistogram = 27, AccessHistogram, uintptr_t)
^
/Users/ec2-user/jenkins/workspace/llvm.org/clang-san-iossim/llvm-project/llvm/lib/ProfileData/MemProfReader.cpp:77:8: note: expanded from macro 'MIBEntryDef'
Io.mapRequired(KeyStr.str().c_str(), MIB.Name); \
^
/Users/ec2-user/jenkins/workspace/llvm.org/clang-san-iossim/llvm-project/llvm/include/llvm/Support/YAMLTraits.h:310:8: note: template is declared here
struct MissingTrait;
^
1 error generated.

show more ...

[memprof] Add YAML-based deserialization for MemProf profile (#117829)

This patch adds YAML-based deserialization for MemProf profile.

It's been painful to write tests for MemProf passes because

[memprof] Add YAML-based deserialization for MemProf profile (#117829)

This patch adds YAML-based deserialization for MemProf profile.

It's been painful to write tests for MemProf passes because we do not
have a text format for the MemProf profile. We would write a test
case in C++, run it for a binary MemProf profile, and then finally run
a test written in LLVM IR with the binary profile.

This patch paves the way toward YAML-based MemProf profile.
Specifically, it adds new class YAMLMemProfReader derived from
MemProfReader. For now, it only adds a function to parse StringRef
pointing to YAML data. Subseqeunt patches will wire it to
llvm-profdata and read from a file.

The field names are based on various printYAML functions in MemProf.h.
I'm not aiming for compatibility with the format used in printYAML,
but I don't see a point in changing the field names.

show more ...

[memprof] Use IndexedMemProfRecord in MemProfReader (NFC) (#117613)

IndexedMemProfRecord contains a complete package of the MemProf
profile, including frames, call stacks, and records. This patch

[memprof] Use IndexedMemProfRecord in MemProfReader (NFC) (#117613)

IndexedMemProfRecord contains a complete package of the MemProf
profile, including frames, call stacks, and records. This patch
replaces the three member variables of MemProfReader with
IndexedMemProfRecord.

This transition significantly simplies both the constructor and the
final "take" method:

MemProfReader(IndexedMemProfData MemProfData)
: MemProfData(std::move(MemProfData)) {}

IndexedMemProfData takeMemProfData() { return std::move(MemProfData); }

show more ...

[memprof] Remove dead code in MemProfReader (NFC) (#117607)

The only constructor in current use is the one that takes
IndexedMemProfData. Likewise, the only accessor in current use is
takeMemProf

[memprof] Remove dead code in MemProfReader (NFC) (#117607)

The only constructor in current use is the one that takes
IndexedMemProfData. Likewise, the only accessor in current use is
takeMemProfData.

show more ...

[memprof] Remove verifyIndexedMemProfRecord and verifyFunctionProfileData (#117412)

This patch removes two functions to verify the consistency between:

- IndexedAllocationInfo::CallStack
- Index

[memprof] Remove verifyIndexedMemProfRecord and verifyFunctionProfileData (#117412)

This patch removes two functions to verify the consistency between:

- IndexedAllocationInfo::CallStack
- IndexedAllocationInfo::CSId

Now that MemProf format Version 1 has been removed,
IndexedAllocationInfo::CallStack doesn't participate in either
serialization or deserialization, so we don't care about the
consistency between the two fields in IndexAllocationInfo.

Subsequent patches will remove uses of the old field and eventually
remove the field.

show more ...

[ProfileData] Remove unused includes (NFC) (#116751)

Identified with misc-include-cleaner.

[memprof] Simplify code with MapVector::operator[] (NFC) (#111335)

Note that the following are all equivalent to each other:

Map.insert({Key, Value()}).first->second
Map.try_emplace(Key).fir

[memprof] Simplify code with MapVector::operator[] (NFC) (#111335)

Note that the following are all equivalent to each other:

Map.insert({Key, Value()}).first->second
Map.try_emplace(Key).first->second
Map[Key]

show more ...

[memprof] Avoid repeated hash lookups (NFC) (#110789)

[Memprof] Fixes memory leak in MemInfoBlock histogram. (#96834)

MemInfoBlocks (MIB) with empty callstacks are erased prematurely from
the CallStackProfileData. This patch frees allocated histogram

[Memprof] Fixes memory leak in MemInfoBlock histogram. (#96834)

MemInfoBlocks (MIB) with empty callstacks are erased prematurely from
the CallStackProfileData. This patch frees allocated histogram buffers
when the MIB is associated with an empty callstack.

show more ...

[Memprof] Fix a warning

This patch fixes:

llvm/lib/ProfileData/MemProfReader.cpp:685:1: error: non-void
function does not return a value in all con trol paths
[-Werror,-Wreturn-type]

While I

[Memprof] Fix a warning

This patch fixes:

llvm/lib/ProfileData/MemProfReader.cpp:685:1: error: non-void
function does not return a value in all con trol paths
[-Werror,-Wreturn-type]

While I am at it, this patch removes an else-after-return.

show more ...

[Memprof] Adds the option to collect AccessCountHistograms for memprof. (#94264)

Adds compile time flag -mllvm -memprof-histogram and runtime flag
histogram=true|false to turn Histogram collection

[Memprof] Adds the option to collect AccessCountHistograms for memprof. (#94264)

Adds compile time flag -mllvm -memprof-histogram and runtime flag
histogram=true|false to turn Histogram collection on and off. The
-memprof-histogram flag relies on -memprof-use-callbacks=true to work.

Updates shadow mapping logic in histogram mode from having one 8 byte
counter for 64 bytes, to 1 byte for 8 bytes, capped at 255. Only
supports this granularity as of now.

Updates the RawMemprofReader and serializing MemoryInfoBlocks to binary
format, including changing to a new version of the raw binary format
from version 3 to version 4.

Updates creating MemoryInfoBlocks with and without Histograms. When two
MemoryInfoBlocks are merged, AccessCounts are summed up and the shorter
Histogram is removed.

Adds a memprof_histogram test case.

Initial commit for adding AccessCountHistograms up until RawProfile for
memprof

show more ...

[ProfileData] Add const to a few places (NFC) (#94803)

[memprof] Use std::unique_ptr instead of std::optional (#94655)

Changing the type of Frame::SymbolName from std::optional<std::string>
to std::unique<std::string> reduces sizeof(Frame) from 64 to 3

[memprof] Use std::unique_ptr instead of std::optional (#94655)

Changing the type of Frame::SymbolName from std::optional<std::string>
to std::unique<std::string> reduces sizeof(Frame) from 64 to 32.

The smaller type reduces the cycle and instruction counts by 23% and
4.4%, respectively, with "llvm-profdata show" modified to deserialize
all MemProfRecords in a MemProf V2 profile. The peak memory usage is
cut down nearly by half.

show more ...

[memprof] Use a SetVector (NFC) (#93312)