1============ 2HLSL Support 3============ 4 5.. contents:: 6 :local: 7 8Introduction 9============ 10 11HLSL Support is under active development in the Clang codebase. This document 12describes the high level goals of the project, the guiding principles, as well 13as some idiosyncrasies of the HLSL language and how we intend to support them in 14Clang. 15 16Project Goals 17============= 18 19The long term goal of this project is to enable Clang to function as a 20replacement for the `DirectXShaderCompiler (DXC) 21<https://github.com/microsoft/DirectXShaderCompiler/>`_ in all its supported 22use cases. Accomplishing that goal will require Clang to be able to process most 23existing HLSL programs with a high degree of source compatibility. 24 25Non-Goals 26--------- 27 28HLSL ASTs do not need to be compatible between DXC and Clang. We do not expect 29identical code generation or that features will resemble DXC's implementation or 30architecture. In fact, we explicitly expect to deviate from DXC's implementation 31in key ways. 32 33Guiding Principles 34================== 35 36This document lacks details for architectural decisions that are not yet 37finalized. Our top priorities are quality, maintainability, and flexibility. In 38accordance with community standards we are expecting a high level of test 39coverage, and we will engineer our solutions with long term maintenance in mind. 40We are also working to limit modifications to the Clang C++ code paths and 41share as much functionality as possible. 42 43Architectural Direction 44======================= 45 46HLSL support in Clang is expressed as C++ minus unsupported C and C++ features. 47This is different from how other Clang languages are implemented. Most languages 48in Clang are additive on top of C. 49 50HLSL is not a formally or fully specified language, and while our goals require 51a high level of source compatibility, implementations can vary and we have some 52flexibility to be more or less permissive in some cases. For modern HLSL DXC is 53the reference implementation. 54 55The HLSL effort prioritizes following similar patterns for other languages, 56drivers, runtimes and targets. Specifically, We will maintain separation between 57HSLS-specific code and the rest of Clang as much as possible following patterns 58in use in Clang code today (i.e. ParseHLSL.cpp, SemaHLSL.cpp, CGHLSL*.cpp...). 59We will use inline checks on language options where the code is simple and 60isolated, and prefer HLSL-specific implementation files for any code of 61reasonable complexity. 62 63In places where the HLSL language is in conflict with C and C++, we will seek to 64make minimally invasive changes guarded under the HLSL language options. We will 65seek to make HLSL language support as minimal a maintenance burden as possible. 66 67DXC Driver 68---------- 69 70A DXC driver mode will provide command-line compatibility with DXC, supporting 71DXC's options and flags. The DXC driver is HLSL-specific and will create an 72HLSLToolchain which will provide the basis to support targeting both DirectX and 73Vulkan. 74 75Parser 76------ 77 78Following the examples of other parser extensions HLSL will add a ParseHLSL.cpp 79file to contain the implementations of HLSL-specific extensions to the Clang 80parser. The HLSL grammar shares most of its structure with C and C++, so we will 81use the existing C/C++ parsing code paths. 82 83Sema 84---- 85 86HLSL's Sema implementation will also provide an ``ExternalSemaSource``. In DXC, 87an ``ExternalSemaSource`` is used to provide definitions for HLSL built-in data 88types and built-in templates. Clang is already designed to allow an attached 89``ExternalSemaSource`` to lazily complete data types, which is a **huge** 90performance win for HLSL. 91 92If precompiled headers are used when compiling HLSL, the ``ExternalSemaSource`` 93will be a ``MultiplexExternalSemaSource`` which includes both the ``ASTReader`` 94and -. For Built-in declarations that are already 95completed in the serialized AST, the ``HLSLExternalSemaSource`` will reuse the 96existing declarations and not introduce new declarations. If the built-in types 97are not completed in the serialized AST, the ``HLSLExternalSemaSource`` will 98create new declarations and connect the de-serialized decls as the previous 99declaration. 100 101CodeGen 102------- 103 104Like OpenCL, HLSL relies on capturing a lot of information into IR metadata. 105*hand wave* *hand wave* *hand wave* As a design principle here we want our IR to 106be idiomatic Clang IR as much as possible. We will use IR attributes wherever we 107can, and use metadata as sparingly as possible. One example of a difference from 108DXC already implemented in Clang is the use of target triples to communicate 109shader model versions and shader stages. 110 111Our HLSL CodeGen implementation should also have an eye toward generating IR 112that will map directly to targets other than DXIL. While IR itself is generally 113not re-targetable, we want to share the Clang CodeGen implementation for HLSL 114with other GPU graphics targets like SPIR-V and possibly other GPU and even CPU 115targets. 116 117hlsl.h 118------ 119 120HLSL has a library of standalone functions. This is similar to OpenCL and CUDA, 121and is analogous to C's standard library. The implementation approach for the 122HLSL library functionality draws from patterns in use by OpenCL and other Clang 123resource headers. All of the clang resource headers are part of the 124``ClangHeaders`` component found in the source tree under 125`clang/lib/Headers <https://github.com/llvm/llvm-project/tree/main/clang/lib/Headers>`_. 126 127.. note:: 128 129 HLSL's complex data types are not defined in HLSL's header because many of 130 the semantics of those data types cannot be expressed in HLSL due to missing 131 language features. Data types that can't be expressed in HLSL are defined in 132 code in the ``HLSLExternalSemaSource``. 133 134Similar to OpenCL, the HLSL library functionality is implicitly declared in 135translation units without needing to include a header to provide declarations. 136In Clang this is handled by making ``hlsl.h`` an implicitly included header 137distributed as part of the Clang resource directory. 138 139Similar to OpenCL, HLSL's implicit header will explicitly declare all overloads, 140and each overload will map to a corresponding ``__builtin*`` compiler intrinsic 141that is handled in ClangCodeGen. CUDA uses a similar pattern although many CUDA 142functions have full definitions in the included headers which in turn call 143corresponding ``__builtin*`` compiler intrinsics. By not having bodies HLSL 144avoids the need for the inliner to clean up and inline large numbers of small 145library functions. 146 147HLSL's implicit headers also define some of HLSL's typedefs. This is consistent 148with how the AVX vector header is implemented. 149 150Concerns have been expressed that this approach may result in slower compile 151times than the approach DXC uses where library functions are treated more like 152Clang ``__builtin*`` intrinsics. No real world use cases have been identified 153where parsing is a significant compile-time overhead, but the HLSL implicit 154headers can be compiled into a module for performance if needed. 155 156Further, by treating these as functions rather than ``__builtin*`` compiler 157intrinsics, the language behaviors are more consistent and aligned with user 158expectation because normal overload resolution rules and implicit conversions 159apply as expected. 160 161It is a feature of this design that clangd-powered "go to declaration" for 162library functions will jump to a valid header declaration and all overloads will 163be user readable. 164 165HLSL Language 166============= 167 168The HLSL language is insufficiently documented, and not formally specified. 169Documentation is available on `Microsoft's website 170<https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl>`_. 171The language syntax is similar enough to C and C++ that carefully written C and 172C++ code is valid HLSL. HLSL has some key differences from C & C++ which we will 173need to handle in Clang. 174 175HLSL is not a conforming or valid extension or superset of C or C++. The 176language has key incompatibilities with C and C++, both syntactically and 177semantically. 178 179An Aside on GPU Languages 180------------------------- 181 182Due to HLSL being a GPU targeted language HLSL is a Single Program Multiple Data 183(SPMD) language relying on the implicit parallelism provided by GPU hardware. 184Some language features in HLSL enable programmers to take advantage of the 185parallel nature of GPUs in a hardware abstracted language. 186 187HLSL also prohibits some features of C and C++ which can have catastrophic 188performance or are not widely supportable on GPU hardware or drivers. As an 189example, register spilling is often excessively expensive on GPUs, so HLSL 190requires all functions to be inlined during code generation, and does not 191support a runtime calling convention. 192 193Pointers & References 194--------------------- 195 196HLSL does not support referring to values by address. Semantically all variables 197are value-types and behave as such. HLSL disallows the pointer dereference 198operators (unary ``*``, and ``->``), as well as the address of operator (unary 199&). While HLSL disallows pointers and references in the syntax, HLSL does use 200reference types in the AST, and we intend to use pointer decay in the AST in 201the Clang implementation. 202 203HLSL ``this`` Keyword 204--------------------- 205 206HLSL does support member functions, and (in HLSL 2021) limited operator 207overloading. With member function support, HLSL also has a ``this`` keyword. The 208``this`` keyword is an example of one of the places where HLSL relies on 209references in the AST, because ``this`` is a reference. 210 211Bitshifts 212--------- 213 214In deviation from C, HLSL bitshifts are defined to mask the shift count by the 215size of the type. In DXC, the semantics of LLVM IR were altered to accommodate 216this, in Clang we intend to generate the mask explicitly in the IR. In cases 217where the shift value is constant, this will be constant folded appropriately, 218in other cases we can clean it up in the DXIL target. 219 220Non-short Circuiting Logical Operators 221-------------------------------------- 222 223In HLSL 2018 and earlier, HLSL supported logical operators (and the ternary 224operator) on vector types. This behavior required that operators not short 225circuit. The non-short circuiting behavior applies to all data types until HLSL 2262021. In HLSL 2021, logical and ternary operators do not support vector types 227instead builtin functions ``and``, ``or`` and ``select`` are available, and 228operators short circuit matching C behavior. 229 230Precise Qualifier 231----------------- 232 233HLSL has a ``precise`` qualifier that behaves unlike anything else in the C 234language. The support for this qualifier in DXC is buggy, so our bar for 235compatibility is low. 236 237The ``precise`` qualifier applies in the inverse direction from normal 238qualifiers. Rather than signifying that the declaration containing ``precise`` 239qualifier be precise, it signifies that the operations contributing to the 240declaration's value be ``precise``. Additionally, ``precise`` is a misnomer: 241values attributed as ``precise`` comply with IEEE-754 floating point semantics, 242and are prevented from optimizations which could decrease *or increase* 243precision. 244 245Differences in Templates 246------------------------ 247 248HLSL uses templates to define builtin types and methods, but disallowed 249user-defined templates until HLSL 2021. HLSL also allows omitting empty template 250parameter lists when all template parameters are defaulted. This is an ambiguous 251syntax in C++, but Clang detects the case and issues a diagnostic. This makes 252supporting the case in Clang minimally invasive. 253 254Vector Extensions 255----------------- 256 257HLSL uses the OpenCL vector extensions, and also provides C++-style constructors 258for vectors that are not supported by Clang. 259 260Standard Library 261---------------- 262 263HLSL does not support the C or C++ standard libraries. Like OpenCL, HLSL 264describes its own library of built in types, complex data types, and functions. 265 266Unsupported C & C++ Features 267---------------------------- 268 269HLSL does not support all features of C and C++. In implementing HLSL in Clang 270use of some C and C++ features will produce diagnostics under HLSL, and others 271will be supported as language extensions. In general, any C or C++ feature that 272can be supported by the DXIL and SPIR-V code generation targets could be treated 273as a clang HLSL extension. Features that cannot be lowered to DXIL or SPIR-V, 274must be diagnosed as errors. 275 276HLSL does not support the following C features: 277 278* Pointers 279* References 280* ``goto`` or labels 281* Variable Length Arrays 282* ``_Complex`` and ``_Imaginary`` 283* C Threads or Atomics (or Obj-C blocks) 284* ``union`` types `(in progress for HLSL 202x) <https://github.com/microsoft/DirectXShaderCompiler/pull/4132>`_ 285* Most features C11 and later 286 287HLSL does not support the following C++ features: 288 289* RTTI 290* Exceptions 291* Multiple inheritance 292* Access specifiers 293* Anonymous or inline namespaces 294* ``new`` & ``delete`` operators in all of their forms (array, placement, etc) 295* Constructors and destructors 296* Any use of the ``virtual`` keyword 297* Most features C++11 and later 298