[Debugify][Original DI] Test dbg var loc preservation

This is an improvement of [0]. This adds checking of
original llvm.dbg.values()/declares() instructions in
optimizations.

We have picked a real

[Debugify][Original DI] Test dbg var loc preservation

This is an improvement of [0]. This adds checking of
original llvm.dbg.values()/declares() instructions in
optimizations.

We have picked a real issue that has been found with
this (actually, picked one variable location missing
from [1] and resolved the issue), and the result is
the fix for that -- D100844.

Before applying the D100844, using the options from [0]
(but with this patch applied) on the compilation of GDB 7.11,
the final HTML report for the debug-info issues can be found
at [1] (please scroll down, and look for
"Summary of Variable Location Bugs"). After applying
the D100844, the numbers has improved a bit -- please take
a look into [2].

[0] https://llvm.org/docs/HowToUpdateDebugInfo.html\
[1] https://djolertrk.github.io/di-check-before-adce-fix/
[2] https://djolertrk.github.io/di-check-after-adce-fix/

Differential Revision: https://reviews.llvm.org/D100845

show more ...

[Transforms][Debugify] Fix "Missing line" false alarm on PHI nodes

This is a fix for https://bugs.llvm.org/show_bug.cgi?id=49959

The "Missing line" false alarm was introduced in D75242.

Patch by Y

[Transforms][Debugify] Fix "Missing line" false alarm on PHI nodes

This is a fix for https://bugs.llvm.org/show_bug.cgi?id=49959

The "Missing line" false alarm was introduced in D75242.

Patch by Yilong Guo<yilong.guo@intel.com>

Differential Revision: https://reviews.llvm.org/D100446

show more ...

[Support] Don't include VirtualFileSystem.h in CommandLine.h

CommandLine.h is indirectly included in ~50% of TUs when building
clang, and VirtualFileSystem.h is large.

(Already remarked by jhenders

[Support] Don't include VirtualFileSystem.h in CommandLine.h

CommandLine.h is indirectly included in ~50% of TUs when building
clang, and VirtualFileSystem.h is large.

(Already remarked by jhenderson on D70769.)

No behavior change.

Differential Revision: https://reviews.llvm.org/D100957

show more ...

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a C

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.

Solution:
This patch adds two new flags

- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.

Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.

So this is the behaviour per platform with my patch:

z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode

Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return

The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```

These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D99426

show more ...

Reapply "[DebugInfo] Handle multiple variable location operands in IR"

Fixed section of code that iterated through a SmallDenseMap and added
instructions in each iteration, causing non-deterministic

Reapply "[DebugInfo] Handle multiple variable location operands in IR"

Fixed section of code that iterated through a SmallDenseMap and added
instructions in each iteration, causing non-deterministic code; replaced
SmallDenseMap with MapVector to prevent non-determinism.

This reverts commit 01ac6d1587e8613ba4278786e8341f8b492ac941.

show more ...

Revert "[DebugInfo] Handle multiple variable location operands in IR"

This caused non-deterministic compiler output; see comment on the
code review.

> This patch updates the various IR passes to co

Revert "[DebugInfo] Handle multiple variable location operands in IR"

This caused non-deterministic compiler output; see comment on the
code review.

> This patch updates the various IR passes to correctly handle dbg.values with a
> DIArgList location. This patch does not actually allow DIArgLists to be produced
> by salvageDebugInfo, and it does not affect any pass after codegen-prepare.
> Other than that, it should cover every IR pass.
>
> Most of the changes simply extend code that operated on a single debug value to
> operate on the list of debug values in the style of any_of, all_of, for_each,
> etc. Instances of setOperand(0, ...) have been replaced with with
> replaceVariableLocationOp, which takes the value that is being replaced as an
> additional argument. In places where this value isn't readily available, we have
> to track the old value through to the point where it gets replaced.
>
> Differential Revision: https://reviews.llvm.org/D88232

This reverts commit df69c69427dea7f5b3b3a4d4564bc77b0926ec88.

show more ...

[Debugify][OriginalDIMode] Export the report into JSON file

By using the original-di check with debugify in the combination with
the llvm/utils/llvm-original-di-preservation.py it becomes very user

[Debugify][OriginalDIMode] Export the report into JSON file

By using the original-di check with debugify in the combination with
the llvm/utils/llvm-original-di-preservation.py it becomes very user
friendly tool. An example of the HTML page with the issues
related to debug info can be found at [0].

[0] https://djolertrk.github.io/di-checker-html-report-example/

Differential Revision: https://reviews.llvm.org/D82546

show more ...

[DebugInfo] Handle multiple variable location operands in IR

This patch updates the various IR passes to correctly handle dbg.values with a
DIArgList location. This patch does not actually allow DIA

[DebugInfo] Handle multiple variable location operands in IR

This patch updates the various IR passes to correctly handle dbg.values with a
DIArgList location. This patch does not actually allow DIArgLists to be produced
by salvageDebugInfo, and it does not affect any pass after codegen-prepare.
Other than that, it should cover every IR pass.

Most of the changes simply extend code that operated on a single debug value to
operate on the list of debug values in the style of any_of, all_of, for_each,
etc. Instances of setOperand(0, ...) have been replaced with with
replaceVariableLocationOp, which takes the value that is being replaced as an
additional argument. In places where this value isn't readily available, we have
to track the old value through to the point where it gets replaced.

Differential Revision: https://reviews.llvm.org/D88232

show more ...

Reland "[Debugify] Make the debugify aware of the original (-g) Debug Info"

As discussed on the RFC [0], I am sharing the set of patches that
enables checking of original Debug Info metadata

Reland "[Debugify] Make the debugify aware of the original (-g) Debug Info"

As discussed on the RFC [0], I am sharing the set of patches that
enables checking of original Debug Info metadata preservation in
optimizations. The proof-of-concept/proposal can be found at [1].

The implementation from the [1] was full of duplicated code,
so this set of patches tries to merge this approach into the existing
debugify utility.

For example, the utility pass in the original-debuginfo-check
mode could be invoked as follows:

$ opt -verify-debuginfo-preserve -pass-to-test sample.ll

Since this is very initial stage of the implementation,
there is a space for improvements such as:
- Add support for the new pass manager
- Add support for metadata other than DILocations and DISubprograms

[0] https://groups.google.com/forum/#!msg/llvm-dev/QOyF-38YPlE/G213uiuwCAAJ
[1] https://github.com/djolertrk/llvm-di-checker

Differential Revision: https://reviews.llvm.org/D82545

The test that was failing is now forced to use the old PM.

show more ...

Revert "[Debugify] Make the debugify aware of the original (-g) Debug Info"

This reverts rG8ee7c7e02953.
One test is failing, I'll reland this as soon as possible.

[Debugify] Make the debugify aware of the original (-g) Debug Info

As discussed on the RFC [0], I am sharing the set of patches that
enables checking of original Debug Info metadata preservation in

[Debugify] Make the debugify aware of the original (-g) Debug Info

As discussed on the RFC [0], I am sharing the set of patches that
enables checking of original Debug Info metadata preservation in
optimizations. The proof-of-concept/proposal can be found at [1].

The implementation from the [1] was full of duplicated code,
so this set of patches tries to merge this approach into the existing
debugify utility.

For example, the utility pass in the original-debuginfo-check
mode could be invoked as follows:

$ opt -verify-debuginfo-preserve -pass-to-test sample.ll

Since this is very initial stage of the implementation,
there is a space for improvements such as:
- Add support for the new pass manager
- Add support for metadata other than DILocations and DISubprograms

[0] https://groups.google.com/forum/#!msg/llvm-dev/QOyF-38YPlE/G213uiuwCAAJ
[1] https://github.com/djolertrk/llvm-di-checker

Differential Revision: https://reviews.llvm.org/D82545

show more ...

[llvm] Construct SmallVector with iterator ranges (NFC)

Reland "[MachineDebugify] Insert synthetic DBG_VALUE instructions"

This reverts commit 841f9c937f6e593c926a26aedf054436eb807fe6.
The change landed many months ago; something else broke those tests.

Revert "[MachineDebugify] Insert synthetic DBG_VALUE instructions"

This reverts commit 2a5675f11d3bc803a245c0e2a3b47491c8f8a065.
The tests it adds fail: https://reviews.llvm.org/D78135#2453736

[Debugify] Skip debugifying on special/immutable passes

With a function pass manager, it would insert debuginfo metadata before
getting to function passes while processing the pass manager, causing

[Debugify] Skip debugifying on special/immutable passes

With a function pass manager, it would insert debuginfo metadata before
getting to function passes while processing the pass manager, causing
debugify to skip while running the function passes.

Skip special passes + verifier + printing passes. Compared to the legacy
implementation of -debugify-each, this additionally skips verifier
passes. Probably no need to update the legacy version since it will be
obsolete soon.

This fixes 2 instcombine tests using -debugify-each under NPM.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D91558

show more ...

[Debugify] Port -debugify-each to NewPM

Preemptively switch 2 tests to the new PM

Reviewed By: aeubanks

Differential Revision: https://reviews.llvm.org/D90365

[Debugify] Move global namespace functions into llvm::

Also move exportDebugifyStats from tools/opt to Debugify.cpp

Revert rG5dd566b7c7b78bd- "PassManager.h - remove unnecessary Function.h/Module.h includes. NFCI."

This reverts commit 5dd566b7c7b78bd385418c72d63c79895be9ae97.

Causing some buildbot failures that

Revert rG5dd566b7c7b78bd- "PassManager.h - remove unnecessary Function.h/Module.h includes. NFCI."

This reverts commit 5dd566b7c7b78bd385418c72d63c79895be9ae97.

Causing some buildbot failures that I'm not seeing on MSVC builds.

show more ...

PassManager.h - remove unnecessary Function.h/Module.h includes. NFCI.

PassManager.h is one of the top headers in the ClangBuildAnalyzer frontend worst offenders list.

This exposes a large number o

PassManager.h - remove unnecessary Function.h/Module.h includes. NFCI.

PassManager.h is one of the top headers in the ClangBuildAnalyzer frontend worst offenders list.

This exposes a large number of implicit dependencies on various forward declarations/includes in other headers that need addressing.

show more ...

[debugify] Demote an error about empty locations to a warning

In https://reviews.llvm.org/D81198, we outlined a number of scenarios
where dropping debug locations is appropriate. Stop issuing an err

[debugify] Demote an error about empty locations to a warning

In https://reviews.llvm.org/D81198, we outlined a number of scenarios
where dropping debug locations is appropriate. Stop issuing an error
when this happens.

show more ...

[NFC][Debugify] Format the CheckModuleDebugify output

This fixes the output of the check-debugify option.
Without the patch an example of running the option:

$ opt -check-debugify test.ll -S -o tes

[NFC][Debugify] Format the CheckModuleDebugify output

This fixes the output of the check-debugify option.
Without the patch an example of running the option:

$ opt -check-debugify test.ll -S -o testDebugify.ll
CheckModuleDebugifySkipping module without debugify metadata

After the patch:

$ opt -check-debugify test.ll -S -o testDebugify.ll
CheckModuleDebugify: Skipping module without debugify metadata

Differential Revision: https://reviews.llvm.org/D80553

show more ...

[Debugify] Do not require named metadata to be present when stripping

This allows -mir-strip-debug to be run without -debugify having run
before.

[MachineDebugify] Insert synthetic DBG_VALUE instructions

Summary:
Teach MachineDebugify how to insert DBG_VALUE instructions. This can
help find bugs causing CodeGen differences when debug info is

[MachineDebugify] Insert synthetic DBG_VALUE instructions

Summary:
Teach MachineDebugify how to insert DBG_VALUE instructions. This can
help find bugs causing CodeGen differences when debug info is present.
DBG_VALUE instructions are only emitted when -debugify-level is set to
locations+variables.

There is essentially no attempt made to match up DBG_VALUE register
operands with the local variables they ought to correspond to. I'm not
sure how to improve the situation. In some cases (MachineMemOperand?)
it's possible to find the IR instruction a MachineInstr corresponds to,
but in general this seems to call for "undoing" the work done by ISel.

Reviewers: dsanders, aprantl

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78135

show more ...

[Debugify] Strip added metadata in the -debugify-each pipeline

Summary:
Share logic to strip debugify metadata between the IR and MIR level
debugify passes. This makes it simpler to hunt for bugs by

[Debugify] Strip added metadata in the -debugify-each pipeline

Summary:
Share logic to strip debugify metadata between the IR and MIR level
debugify passes. This makes it simpler to hunt for bugs by diffing IR
with vs. without -debugify-each turned on.

As a drive-by, fix an issue causing CallGraphNodes to become invalid
when a dead llvm.dbg.value prototype is deleted.

Reviewers: dsanders, aprantl

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77915

show more ...

Add MIR-level debugify with only locations support for now

Summary:
Re-used the IR-level debugify for the most part. The MIR-level code then
adds locations to the MachineInstrs afterwards based on t

Add MIR-level debugify with only locations support for now

Summary:
Re-used the IR-level debugify for the most part. The MIR-level code then
adds locations to the MachineInstrs afterwards based on the LLVM-IR debug
info.

It's worth mentioning that the resulting locations make little sense as
the range of line numbers used in a Function at the MIR level exceeds that
of the equivelent IR level function. As such, MachineInstrs can appear to
originate from outside the subprogram scope (and from other subprogram
scopes). However, it doesn't seem worth worrying about as the source is
imaginary anyway.

There's a few high level goals this pass works towards:
* We should be able to debugify our .ll/.mir in the lit tests without
changing the checks and still pass them. I.e. Debug info should not change
codegen. Combining this with a strip-debug pass should enable this. The
main issue I ran into without the strip-debug pass was instructions with MMO's and
checks on both the instruction and the MMO as the debug-location is
between them. I currently have a simple hack in the MIRPrinter to
resolve that but the more general solution is a proper strip-debug pass.
* We should be able to test that GlobalISel does not lose debug info. I
recently found that the legalizer can be unexpectedly lossy in seemingly
simple cases (e.g. expanding one instr into many). I have a verifier
(will be posted separately) that can be integrated with passes that use
the observer interface and will catch location loss (it does not verify
correctness, just that there's zero lossage). It is a little conservative
as the line-0 locations that arise from conflicts do not track the
conflicting locations but it can still catch a fair bit.

Depends on D77439, D77438

Reviewers: aprantl, bogner, vsk

Subscribers: mgorny, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77446

show more ...