[WebAssembly] Fix catch block type in wasm64 (#124381)

`try_table`'s `catch` or `catch_ref`'s target block's return type should
be `i64` and `(i64, exnref)` in case of wasm64.

[WebAssembly] Add unreachable before catch destinations (#123915)

When `try_table`'s catch clause's destination has a return type, as in
the case of catch with a concrete tag, catch_ref, and catch_

[WebAssembly] Add unreachable before catch destinations (#123915)

When `try_table`'s catch clause's destination has a return type, as in
the case of catch with a concrete tag, catch_ref, and catch_all_ref. For
example:
```wasm
block exnref
try_table (catch_all_ref 0)
...
end_try_table
end_block
... use exnref ...
```

This code is not valid because the block's body type is not exnref. So
we add an unreachable after the 'end_try_table' to make the code valid
here:
```wasm
block exnref
try_table (catch_all_ref 0)
...
end_try_table
unreachable ;; Newly added
end_block
```
Because 'unreachable' is a terminator we also need to split the BB.

---

We need to handle the same thing for unwind mismatch handling. In the
code below, we create a "trampoline BB" that will be the destination for
the nested `try_table`~`end_try_table` added to fix a unwind mismatch:
```wasm
try_table (catch ... )
block exnref
...
try_table (catch_all_ref N)
some code
end_try_table
...
end_block ;; Trampoline BB
throw_ref
end_try_table
```
While the `block` added for the trampoline BB has the return type
`exnref`, its body, which contains the nested `try_table` and other
code, wouldn't have the `exnref` return type. Most times it didn't
become a problem because the block's body ended with something like `br`
or `return`, but that may not always be the case, especially when there
is a loop. So we add an `unreachable` to make the code valid here too:
```wasm
try_table (catch ... )
block exnref
...
try_table (catch_all_ref N)
some code
end_try_table
...
unreachable ;; Newly added
end_block ;; Trampoline BB
throw_ref
end_try_table
```
In this case we just append the `unreachable` at the end of the layout
predecessor BB. (This was tricky to do in the first (non-mismatch) case
because there `end_try_table` and `end_block` were added in the
beginning of an EH pad in `placeTryTableMarker` and moving
`end_try_table` and the new `unreachable` to the previous BB caused
other problems.)

---

This adds many `unreaachable`s to the output, but this adds
`unreachable` to only a few places to see if this is working. The
FileCheck lines in `exception.ll` and `cfg-stackify-eh.ll` are already
heavily redacted to only leave important control-flow instructions, so I
don't think it's worth adding `unreachable`s everywhere.

show more ...

[WebAssembly] Add -wasm-use-legacy-eh option (#122158)

This replaces the existing `-wasm-enable-exnref` with
`-wasm-use-legacy-eh` option, in an effort to make the new standardized
exnref proposal

[WebAssembly] Add -wasm-use-legacy-eh option (#122158)

This replaces the existing `-wasm-enable-exnref` with
`-wasm-use-legacy-eh` option, in an effort to make the new standardized
exnref proposal the 'default' state and the legacy proposal needs to be
separately enabled an option. But given that most users haven't switched
to the new proposal and major web browsers haven't turned it on by
default, this `-wasm-use-legacy-eh` is turned on by default, so nothing
will change for now for the functionality perspective.

This also removes the restriction that `-wasm-enable-exnref` be only
used with `-wasm-enable-eh` because this option is enabled by default.
This option does not have any effect when `-wasm-enable-eh` is not used.

show more ...

[WebAssembly] Fix rethrow's index calculation (#114693)

So far we have assumed that we only rethrow the exception caught in the
innermost EH pad. This is true in code we directly generate, but afte

[WebAssembly] Fix rethrow's index calculation (#114693)

So far we have assumed that we only rethrow the exception caught in the
innermost EH pad. This is true in code we directly generate, but after
inlining this may not be the case. For example, consider this code:
```ll
ehcleanup:
%0 = cleanuppad ...
call @destructor
cleanupret from %0 unwind label %catch.dispatch
```

If `destructor` gets inlined into this function, the code can be like
```ll
ehcleanup:
%0 = cleanuppad ...
invoke @throwing_func
to label %unreachale unwind label %catch.dispatch.i

catch.dispatch.i:
catchswitch ... [ label %catch.start.i ]

catch.start.i:
%1 = catchpad ...
invoke @some_function
to label %invoke.cont.i unwind label %terminate.i

invoke.cont.i:
catchret from %1 to label %destructor.exit

destructor.exit:
cleanupret from %0 unwind label %catch.dispatch
```

We lower a `cleanupret` into `rethrow`, which assumes it rethrows the
exception caught by the nearest dominating EH pad. But after the
inlining, the nearest dominating EH pad is not `ehcleanup` but
`catch.start.i`.

The problem exists in the same manner in the new (exnref) EH, because it
assumes the exception comes from the nearest EH pad and saves an exnref
from that EH pad and rethrows it (using `throw_ref`).

This problem can be fixed easily if `cleanupret` has the basic block
where its matching `cleanuppad` is. The bitcode instruction `cleanupret`
kind of has that info (it has a token from the `cleanuppad`), but that
info is lost when when we enter ISel, because `TargetSelectionDAG.td`'s
`cleanupret` node does not have any arguments:

https://github.com/llvm/llvm-project/blob/5091a359d9807db8f7d62375696f93fc34226969/llvm/include/llvm/Target/TargetSelectionDAG.td#L700
Note that `catchret` already has two basic block arguments, even though
neither of them means `catchpad`'s BB.

This PR adds the `cleanuppad`'s BB as an argument to `cleanupret` node
in ISel and uses it in the Wasm backend. Because this node is also used
in X86 backend we need to note its argument there too but nothing more
needs to change there as long as X86 doesn't need it.

---

- Details about changes in the Wasm backend:

After this PR, our pseudo `RETHROW` instruction takes a BB, which means
the EH pad whose exception it needs to rethrow. There are currently two
ways to generate a `RETHROW`: one is from `llvm.wasm.rethrow` intrinsic
and the other is from `CLEANUPRET` we discussed above. In case of
`llvm.wasm.rethrow`, we add a '0' as a placeholder argument when it is
lowered to a `RETHROW`, and change it to a BB in LateEHPrepare. As
written in the comments, this PR doesn't change how this BB is computed.
The BB argument will be converted to an immediate argument as with other
control flow instructions in CFGStackify.

In case of `CLEANUPRET`, it already has a BB argument pointing to an EH
pad, so it is just converted to a `RETHROW` with the same BB argument in
LateEHPrepare. This will also be lowered to an immediate in CFGStackify
with other control flow instructions.

---

Fixes #114600.

show more ...

[WebAssembly] Fix unwind mismatches in new EH (#114361)

This fixes unwind mismatches for the new EH spec.

The main flow is similar to that of the legacy EH's unwind mismatch
fixing. The new EH s

[WebAssembly] Fix unwind mismatches in new EH (#114361)

This fixes unwind mismatches for the new EH spec.

The main flow is similar to that of the legacy EH's unwind mismatch
fixing. The new EH shared `fixCallUnwindMismatches` and
`fixCatchUnwindMismatches` functions, which gather the range of
instructions we need to fix their unwind destination for, with the
legacy EH. But unlike the legacy EH that uses `try`-`delegate`s to fix
them, the new EH wrap those instructions with nested
`try_table`-`end_try_table`s that jump to a "trampoline" BB, where we
rethrow (using a `throw_ref`) the exception to the correct `try_table`.

For a simple example of a call unwind mismatch, suppose if `call foo`
should unwind to the outer `try_table` but is wrapped in another
`try_table` (not shown here):
```wast
try_table
...
call foo ;; Unwind mismatch. Should unwind to the outer try_table
...
end_try_table
```

Then we wrap the call with a new nested `try_table`-`end_try_table`, add
a `block` / `end_block` right inside the target `try_table`, and make
the nested `try_table` jump to it using a `catch_all_ref` clause, and
rethrow the exception using a `throw_ref`:
```wast
try_table
block $l0 exnref
...
try_table (catch_all_ref $l0)
call foo
end_try_table
...
end_block ;; Trampoline BB
throw_ref
end_try_table
```

---

This fixes two existing bugs. These are not easy to test independently
without the unwind mismatch fixing. The first one is how we calculate
`ScopeTops`. Turns out, we should do it in the same way as in the legacy
EH even though there is no `end_try` at the end of `catch` block
anymore. `nested_try` in `cfg-stackify-eh.ll` tests this case.

The second bug is in `rewriteDepthImmediates`. `try_table`'s immediates
should be computed without the `try_table` itself, meaning
```wast
block
try_table (catch ... 0)
end_try_table
end_block
```
Here 0 should target not `end_try_table` but `end_block`. This bug
didn't crash the program because `placeTryTableMarker` generated only
the simple form of `try_table` that has a single catch clause and an
`end_block` follows right after the `end_try_table` in the same BB, so
jumping to an `end_try_table` is the same as jumping to the `end_block`.
But now we generate `catch` clauses with depths greater than 0 with when
fixing unwind mismatches, which uncovered this bug.

---

One case that needs a special treatment was when `end_loop` precedes an
`end_try_table` within a BB and this BB is a (true) unwind destination
when fixing unwind mismatches. In this case we need to split this
`end_loop` into a predecessor BB. This case is tested in
`unwind_mismatches_with_loop` in `cfg-stackify-eh.ll`.

---

`cfg-stackify-eh.ll` contains mostly the same set of tests with the
existing `cfg-stackify-eh-legacy.ll` with the updated FileCheck
expectations. As in `cfg-stackify-eh-legacy.ll`, the FileCheck lines
mostly only contain control flow instructions and calls for readability.
- `nested_try` and `unwind_mismatches_with_loop` are added to test newly
found bugs in the new EH.
- Some tests in `cfg-stackify-eh-legacy.ll` about the legacy-EH-specific
asepcts have not been added to `cfg-stackify-eh.ll`.
(`remove_unnecessary_instrs`, `remove_unnecessary_br`,
`fix_function_end_return_type_with_try_catch`, and
`branch_remapping_after_fixing_unwind_mismatches_0/1`)

show more ...

[WebAssembly] Fix comments in CFGStackify (NFC) (#114362)

This fixes comments in CFGStackify and renames a variable to be clearer.

[WebAssembly] Add assembly support for final EH proposal (#107917)

This adds the basic assembly generation support for the final EH
proposal, which was newly adopted in Sep 2023 and advanced into P

[WebAssembly] Add assembly support for final EH proposal (#107917)

This adds the basic assembly generation support for the final EH
proposal, which was newly adopted in Sep 2023 and advanced into Phase 4
in Jul 2024:

https://github.com/WebAssembly/exception-handling/blob/main/proposals/exception-handling/Exceptions.md

This adds support for the generation of new `try_table` and `throw_ref`
instruction in .s asesmbly format. This does NOT yet include
- Block annotation comment generation for .s format
- .o object file generation
- .s assembly parsing
- Type checking (AsmTypeCheck)
- Disassembler
- Fixing unwind mismatches in CFGStackify

These will be added as follow-up PRs.

---

The format for `TRY_TABLE`, both for `MachineInstr` and `MCInst`, is as
follows:
```
TRY_TABLE type number_of_catches catch_clauses*
```
where `catch_clause` is
```
catch_opcode tag+ destination
```
`catch_opcode` should be one of 0/1/2/3, which denotes
`CATCH`/`CATCH_REF`/`CATCH_ALL`/`CATCH_ALL_REF` respectively. (See
`BinaryFormat/Wasm.h`) `tag` exists when the catch is one of `CATCH` or
`CATCH_REF`.
The MIR format is printed as just the list of raw operands. The
(stack-based) assembly instruction supports pretty-printing, including
printing `catch` clauses by name, in InstPrinter.

In addition to the new instructions `TRY_TABLE` and `THROW_REF`, this
adds four pseudo instructions: `CATCH`, `CATCH_REF`, `CATCH_ALL`, and
`CATCH_ALL_REF`. These are pseudo instructions to simulate block return
values of `catch`, `catch_ref`, `catch_all`, `catch_all_ref` clauses in
`try_table` respectively, given that we don't support block return
values except for one case (`fixEndsAtEndOfFunction` in CFGStackify).
These will be omitted when we lower the instructions to `MCInst` at the
end.

LateEHPrepare now will have one more stage to covert
`CATCH`/`CATCH_ALL`s to `CATCH_REF`/`CATCH_ALL_REF`s when there is a
`RETHROW` to rethrow its exception. The pass also converts `RETHROW`s
into `THROW_REF`. Note that we still use `RETHROW` as an interim pseudo
instruction until we convert them to `THROW_REF` in LateEHPrepare.

CFGStackify has a new `placeTryTableMarker` function, which places
`try_table`/`end_try_table` markers with a necessary `catch` clause and
also `block`/`end_block` markers for the destination of the `catch`
clause.

In MCInstLower, now we need to support one more case for the multivalue
block signature (`catch_ref`'s destination's `(i32, exnref)` return
type).

InstPrinter has a new routine to print the `catch_list` type, which is
used to print `try_table` instructions.

The new test, `exception.ll`'s source is the same as
`exception-legacy.ll`, with the FileCheck expectations changed. One
difference is the commands in this file have `-wasm-enable-exnref` to
test the new format, and don't have `-wasm-disable-explicit-locals
-wasm-keep-registers`, because the new custom InstPrinter routine to
print `catch_list` only works for the stack-based instructions (`_S`),
and we can't use `-wasm-keep-registers` for them.

As in `exception-legacy.ll`, the FileCheck lines for the new tests do
not contain the whole program; they mostly contain only the control flow
instructions for readability.

show more ...

[WebAssembly] Simplify a switch-case in CFGStackify (NFC) (#107360)

This merges some `case`s using `[[fallthrough]]`, and make `DELEGATE` as
a separate `case`. (Previously the reason we didn't do t

[WebAssembly] Simplify a switch-case in CFGStackify (NFC) (#107360)

This merges some `case`s using `[[fallthrough]]`, and make `DELEGATE` as
a separate `case`. (Previously the reason we didn't do that was not to
duplicate the code in `RewriteOperands`. But now that we've extracted it
into a lambda function in #107182 we can do it.

show more ...

[WebAssembly] Rename CATCH/CATCH_ALL to *_LEGACY (#107187)

This renames MIR instruction `CATCH` and `CATCH_ALL` to `CATCH_LEGACY`
and `CATCH_ALL_LEGACY` respectively.

Follow-up PRs for the new E

[WebAssembly] Rename CATCH/CATCH_ALL to *_LEGACY (#107187)

This renames MIR instruction `CATCH` and `CATCH_ALL` to `CATCH_LEGACY`
and `CATCH_ALL_LEGACY` respectively.

Follow-up PRs for the new EH (exnref) implementation will use `CATCH`,
`CATCH_REF`, `CATCH_ALL`, and `CATCH_ALL_REF` as pseudo-instructions
that return extracted values or `exnref` or both, because we don't
currently support block return values in LLVM. So to give the old (real)
`CATCH`es and the new (pseudo) `CATCH`es different names, this attaches
`_LEGACY` prefix to the old names.

This also rearranges `WebAssemblyInstrControl.td` so that the old legacy
instructions are listed all together at the end.

show more ...

[WebAssembly] Misc. fixes in CFGStackify (NFC) (#107182)

This contains misc. small fixes in CFGStackify. Most of them are comment
fixes and variable name changes. Two code changes are removing the

[WebAssembly] Misc. fixes in CFGStackify (NFC) (#107182)

This contains misc. small fixes in CFGStackify. Most of them are comment
fixes and variable name changes. Two code changes are removing the cases
that can never occur. Another is extracting a routine as a lambda
function. I will add explanations inline in the code as Github comments.

show more ...

[CodeGen] Use optimized domtree for MachineFunction (#102107)

The dominator tree gained an optimization to use block numbers instead
of a DenseMap to store blocks. Given that machine basic blocks a

[CodeGen] Use optimized domtree for MachineFunction (#102107)

The dominator tree gained an optimization to use block numbers instead
of a DenseMap to store blocks. Given that machine basic blocks already
have numbers, expose these via appropriate GraphTraits. For debugging,
block number epochs are added to MachineFunction -- this greatly helps
in finding uses of block numbers after RenumberBlocks().

In a few cases where dominator trees are preserved across renumberings,
the dominator tree is updated to use the new numbers.

show more ...

[CodeGen][NewPM] Port `machine-loops` to new pass manager (#97793)

- Add `MachineLoopAnalysis`.
- Add `MachineLoopPrinterPass`.
- Convert to `MachineLoopInfoWrapperPass` in legacy pass manager.

[CodeGen][NewPM] Split `MachineDominatorTree` into a concrete analysis result (#94571)

Prepare for new pass manager version of `MachineDominatorTreeAnalysis`.
We may need a machine dominator tree v

[CodeGen][NewPM] Split `MachineDominatorTree` into a concrete analysis result (#94571)

Prepare for new pass manager version of `MachineDominatorTreeAnalysis`.
We may need a machine dominator tree version of `DomTreeUpdater` to
handle `SplitCriticalEdge` in some CodeGen passes.

show more ...

Ensure NoTrapAfterNoreturn is false for the wasm backend (#65876)

In the WebAssembly back end, the TrapUnreachable option is currently
load-bearing for correctness, inserting wasm `unreachable` ins

Ensure NoTrapAfterNoreturn is false for the wasm backend (#65876)

In the WebAssembly back end, the TrapUnreachable option is currently
load-bearing for correctness, inserting wasm `unreachable` instructions
where needed to create valid wasm. There is another option,
NoTrapAfterNoreturn, that removes some of those traps and causes
incorrect wasm to be emitted.

This turns off `NoTrapAfterNoreturn` for the Wasm backend and adds new
tests.

show more ...

[WebAssembly] Create separation between MC and CodeGen layers

Move WebAssemblyUtilities from Utils to the CodeGen library. It
primarily deals in MIR layer types, so it really lives in the CodeGen
li

[WebAssembly] Create separation between MC and CodeGen layers

Move WebAssemblyUtilities from Utils to the CodeGen library. It
primarily deals in MIR layer types, so it really lives in the CodeGen
library.

Move a variety of other things around to try create better separation.

See issue #64166 for more info on layering.

Move llvm/include/CodeGen/WasmAddressSpaces.h back to
llvm/lib/Target/WebAssembly/Utils.

Differential Revision: https://reviews.llvm.org/D156472

show more ...

[WebAssembly] Error out on invalid personality functions

Without explicitly checking and erroring out, an invalid personality
function, which is not `__gxx_wasm_personality_v0`, caused
a segmentatio

[WebAssembly] Error out on invalid personality functions

Without explicitly checking and erroring out, an invalid personality
function, which is not `__gxx_wasm_personality_v0`, caused
a segmentation fault down the line because `WasmEHFuncInfo` was not
created. This explicitly checks the validity of personality functions in
functions with EH pads and errors out explicitly with a helpful error
message. This also adds some more assertions to ensure `WasmEHFuncInfo`
is correctly created and non-null.

Invalid personality functions wouldn't be generated by our Clang, but
can be present in handwritten ll files, and more often, in files
transformed by passes like `metarenamer`, which is often used with
`bugpoint` to simplify names in `bugpoint`-reduced files.

Reviewed By: dschuff

Differential Revision: https://reviews.llvm.org/D152203

show more ...

[CodeGen][Target] Remove uses of Register::isPhysicalRegister/isVirtualRegister. NFC

Use isPhysical/isVirtual methods.

[llvm] LLVM_FALLTHROUGH => [[fallthrough]]. NFC

With C++17 there is no Clang pedantic warning or MSVC C5051.

[WebAssembly][NFC] Consolidate TargetRegisterClass=>COPY opcode conversion into a single helper

Previously WebAssemblyCFGStackify, WebAssemblyInstrInfo, and
WebAssemblyPeephole all had equivalent lo

[WebAssembly][NFC] Consolidate TargetRegisterClass=>COPY opcode conversion into a single helper

Previously WebAssemblyCFGStackify, WebAssemblyInstrInfo, and
WebAssemblyPeephole all had equivalent logic for this. Move it into a
common helper in WebAssemblyUtilities.

show more ...

[NFC][CodeGen] Rename some functions in MachineInstr.h and remove duplicated comments

[NFC] Rename MachineFunction::DeleteMachineBasicBlock

Renamed to conform to coding style

[NFC][llvm] Inclusive language: reword and remove uses of sanity in llvm/lib/Target

Reworded removed code comments that contain `sanity check` and `sanity
test`.

[Target] Use llvm::reverse (NFC)

[llvm] Use make_early_inc_range (NFC)

[WebAssembly] Fix fixEndsAtEndOfFunction for delegate

Background:
CFGStackify's [[ https://github.com/llvm/llvm-project/blob/398f25340000f26d648ebbc7eae9dc401ffc7d5f/llvm/lib/Target/WebAssembly/WebA

[WebAssembly] Fix fixEndsAtEndOfFunction for delegate

Background:
CFGStackify's [[ https://github.com/llvm/llvm-project/blob/398f25340000f26d648ebbc7eae9dc401ffc7d5f/llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp#L1481-L1540 | fixEndsAtEndOfFunction ]] fixes block/loop/try's return
type when the end of function is unreachable and the function return
type is not void. So if a function returns i32 and `block`-`end` wraps the
whole function, i.e., the `block`'s `end` is the last instruction of the
function, the `block`'s return type should be i32 too:
```
block i32
...
end
end_function
```

If there are consecutive `end`s, this signature has to be propagate to
those blocks too, like:
```
block i32
...
block i32
...
end
end
end_function
```

This applies to `try`-`end` too:
```
try i32
...
catch
...
end
end_function
```

In case of `try`, we not only follow consecutive `end`s but also follow
`catch`, because for the type of the whole `try` to be i32, both `try`
and `catch` parts have to be i32:
```
try i32
...
block i32
...
end
catch
...
block i32
...
end
end
end_function
```

---

Previously we only handled consecutive `end`s or `end` before a `catch`.
But now we have `delegate`, which serves like `end` for
`try`-`delegate`. So we have to follow `delegate` too and mark its
corresponding `try` as i32 (the function's return type):
```
try i32
...
catch
...
try i32 ;; Here
...
delegate N
end
end_function
```

Reviewed By: tlively

Differential Revision: https://reviews.llvm.org/D101036

show more ...