1=============================================== 2Architecture and Design of DXIL Support in LLVM 3=============================================== 4 5.. contents:: 6 :local: 7 8.. toctree:: 9 :hidden: 10 11Introduction 12============ 13 14LLVM supports reading and writing the `DirectX Intermediate Language. 15<https://github.com/microsoft/DirectXShaderCompiler/blob/main/docs/DXIL.rst>`_, 16or DXIL. DXIL is essentially LLVM 3.7 era bitcode with some 17restrictions and various semantically important operations and 18metadata. 19 20LLVM's implementation philosophy for DXIL support is to treat DXIL as 21merely a representation format as much as possible. When reading DXIL, 22we should translate everything to generic LLVM constructs when 23possible. Similarly, we should introduce DXIL-specific constructs as 24late as possible in the process of lowering to the format. 25 26There are three places to look for DXIL related code in LLVM: The 27`DirectX` backend, for writing DXIL; The `DXILUpgrade` pass, for 28reading; and in library code that is shared between writing and 29reading. We'll describe these in reverse order. 30 31Common Code for Reading and Writing 32=================================== 33 34There's quite a bit of logic that needs to be shared between reading 35and writing DXIL in order to avoid code duplication. While we don't 36have a hard and fast rule about where such code should live, there are 37generally three sensible places. Simple definitions of enums and 38values that must stay fixed to match DXIL's ABI can be found in 39`Support/DXILABI.h`, utilities to translate bidirectionally between 40DXIL and modern LLVM constructs live in `lib/Transforms/Utils`, and 41more analyses that are needed to derive or preserve information are 42implemented as typical `lib/Analysis` passes. 43 44The DXILUpgrade Pass 45==================== 46 47Translating DXIL to LLVM IR takes advantage of the fact that DXIL is 48compatible with LLVM 3.7 bitcode, and that modern LLVM is capable of 49"upgrading" older bitcode into modern IR. Simply relying on the 50bitcode upgrade process isn't sufficient though, since that leaves a 51number of DXIL specific constructs around. Thus, we have the 52`DXILUpgrade` pass to transform DXIL operations to LLVM operations and 53smooth over differences in metadata representation. We call this pass 54"upgrade" to reflect that it follows LLVM's standard bitcode upgrade 55process and simply finishes the job for DXIL constructs - while 56"reader" or "lifting" might also be reasonable names, they could be a 57bit misleading. 58 59The `DXILUpgrade` pass itself is fairly lightweight. It mostly relies 60on the utilities described in "Common Code" above in order to share 61logic with both the DirectX backend and with Clang's codegen of HLSL 62support as much as possible. 63 64The DirectX Intrinsic Expansion Pass 65==================================== 66There are intrinsics that don't map directly to DXIL Ops. In some cases 67an intrinsic needs to be expanded to a set of LLVM IR instructions. In 68other cases an intrinsic needs modifications to the arguments or return 69values of a DXIL Op. The `DXILIntrinsicExpansion` pass handles all 70the cases where our intrinsics don't have a one to one mapping. This 71pass may also be used when the expansion is specific to DXIL to keep 72implementation details out of CodeGen. Finally, there is an expectation 73that we maintain vector types through this pass. Therefore, best 74practice would be to avoid scalarization in this pass. 75 76 77The DirectX Backend 78=================== 79 80The DirectX backend lowers LLVM IR into DXIL. As we're transforming to 81an intermediate format rather than a specific ISA, this backend does 82not follow the instruction selection patterns you might be familiar 83with from other backends. There are two parts to lowering DXIL - a set 84of passes that mutate various constructs into a form that matches how 85DXIL represents those constructs, followed by a limited bitcode 86"downgrader pass". 87 88Before emitting DXIL, the DirectX backend needs to modify the LLVM IR 89such that external operations, types, and metadata is represented in 90the way that DXIL expects. For example, `DXILOpLowering` translates 91intrinsics into `dx.op` calls. These passes are essentially the 92inverse of the `DXILUpgrade` pass. It's best to do this downgrading 93process as IR to IR passes when possible, as that means that they can 94be easily tested with `opt` and `FileCheck` without the need for 95external tooling. 96 97The second part of DXIL emission is more or less an LLVM bitcode 98downgrader. We need to emit bitcode that matches the LLVM 3.7 99representation. For this, we have `DXILWriter`, which is an alternate 100version of LLVM's `BitcodeWriter`. At present, this is able to 101leverage LLVM's current bitcode libraries to do a lot of the work, but 102it's possible that at some point in the future it will need to be 103completely separate as modern LLVM bitcode evolves. 104 105DirectX Backend Flow 106-------------------- 107 108The code generation flow for DXIL is broken into a series of passes. The passes 109are grouped into two flows: 110 111#. Generating DXIL IR. 112#. Generating DXIL Binary. 113 114The passes to generate DXIL IR follow the flow: 115 116 DXILOpLowering -> DXILPrepare -> DXILTranslateMetadata 117 118Each of these passes has a defined responsibility: 119 120#. DXILOpLowering translates LLVM intrinsic calls to dx.op calls. 121#. DXILPrepare transforms the DXIL IR to be compatible with LLVM 3.7, and 122 inserts bitcasts to allow typed pointers to be inserted. 123#. DXILTranslateMetadata emits the DXIL Metadata structures. 124 125The passes to encode DXIL to binary in the DX Container follow the flow: 126 127 DXILEmbedder -> DXContainerGlobals -> AsmPrinter 128 129Each of these passes have the following defined responsibilities: 130 131#. DXILEmbedder runs the DXIL bitcode writer to generate a bitcode stream and 132 embeds the binary data inside a global in the original module. 133#. DXContainerGlobals generates binary data globals for the other DX Container 134 parts based on computed analysis passes. 135#. AsmPrinter is the standard LLVM infrastructure for emitting object files. 136 137When emitting DXIL into a DX Container file the MC layer is used in a similar 138way to how the Clang ``-fembed-bitcode`` option operates. The DX Container 139object writer knows how to construct the headers and structural fields of the 140container, and reads global variables from the module to fill in the remaining 141part data. 142 143DirectX Container 144----------------- 145 146The DirectX container format is treated in LLVM as an object file format. 147Reading is implemented between the BinaryFormat and Object libraries, and 148writing is implemented in the MC layer. Additional testing and inspection 149support are implemented in the ObjectYAML library and tools. 150 151Testing 152======= 153 154A lot of DXIL testing can be done with typical IR to IR tests using 155`opt` and `FileCheck`, since a lot of the support is implemented in 156terms of IR level passes as described in the previous sections. You 157can see examples of this in `llvm/test/CodeGen/DirectX` as well as 158`llvm/test/Transforms/DXILUpgrade`, and this type of testing should be 159leveraged as much as possible. 160 161However, when it comes to testing the DXIL format itself, IR passes 162are insufficient for testing. For now, the best option we have 163available is using the DXC project's tools in order to round trip. 164These tests are currently found in `test/tools/dxil-dis` and are only 165available if the `LLVM_INCLUDE_DXIL_TESTS` cmake option is set. Note 166that we do not currently have the equivalent testing set up for the 167DXIL reading path. 168 169As soon as we are able, we will also want to round trip using the DXIL 170writing and reading paths in order to ensure self consistency and to 171get test coverage when `dxil-dis` isn't available. 172