1# MLIR : Language Server Protocol 2 3[TOC] 4 5This document describes the tools and utilities related to supporting 6[LSP](https://microsoft.github.io/language-server-protocol/) IDE language 7extensions for various MLIR-related languages. An LSP language extension is 8generally comprised of two components; a language client and a language server. 9A language client is a piece of code that interacts with the IDE that you are 10using, such as VSCode. A language server acts as the backend for queries that 11the client may want to perform, such as "Find Definition", "Find References", 12etc. 13 14## MLIR LSP Language Server : `mlir-lsp-server` 15 16MLIR provides an implementation of an LSP language server for `.mlir` text files 17in the form of the `mlir-lsp-server` tool. This tool interacts with the MLIR C++ 18API to support rich language queries, such as "Find Definition". 19 20### Supporting custom dialects 21 22`mlir-lsp-server`, like many other MLIR based tools, relies on having the 23appropriate dialects registered to be able to parse in the custom assembly 24formats used in the textual .mlir files. The `mlir-lsp-server` found within the 25main MLIR repository provides support for all of the upstream MLIR dialects. 26Downstream and out-of-tree users will need to provide a custom 27`mlir-lsp-server` executable that registers the entities that they are 28interested in. The implementation of `mlir-lsp-server` is provided as a library, 29making it easy for downstream users to register their dialect and simply 30call into the main implementation. A simple example is shown below: 31 32```c++ 33#include "mlir/Tools/mlir-lsp-server/MlirLspServerMain.h" 34 35int main(int argc, char **argv) { 36 mlir::DialectRegistry registry; 37 registerMyDialects(registry); 38 return mlir::failed(mlir::MlirLspServerMain(argc, argv, registry)); 39} 40``` 41 42See the [Editor Plugins](#editor-plugins) section below for details on how to 43setup support in a few known LSP clients, such as vscode. 44 45### Features 46 47This section details a few of the features that the MLIR language server 48provides. The screenshots are shown in [VSCode](https://code.visualstudio.com/), 49but the exact feature set available will depend on your editor client. 50 51[mlir features]: # 52 53#### Diagnostics 54 55The language server actively runs verification on the IR as you type, showing 56any generated diagnostics in-place. 57 58 59 60##### Automatically insert `expected-` diagnostic checks 61 62MLIR provides 63[infrastructure](https://mlir.llvm.org/docs/Diagnostics/#sourcemgr-diagnostic-verifier-handler) 64for checking expected diagnostics, which is heavily utilized when defining IR 65parsing and verification. The language server provides code actions for 66automatically inserting the checks for diagnostics it knows about. 67 68 69 70#### Code completion 71 72The language server provides suggestions as you type, offering completions for 73dialect constructs (such as attributes, operations, and types), block names, SSA 74value names, keywords, and more. 75 76 77 78#### Cross-references 79 80Cross references allow for navigating the use/def chains of SSA values (i.e. 81operation results and block arguments), [Symbols](../SymbolsAndSymbolTables.md), 82and Blocks. 83 84##### Find definition 85 86Jump to the definition of the IR entity under the cursor. A few examples are 87shown below: 88 89- SSA Values 90 91 92 93- Symbol References 94 95 96 97The definition of an operation will also take into account the source location 98attached, allowing for navigating into the source file that generated the 99operation. 100 101 102 103##### Find references 104 105Show all references of the IR entity under the cursor. 106 107 108 109#### Hover 110 111Hover over an IR entity to see more information about it. The exact information 112displayed is dependent on the type of IR entity under the cursor. For example, 113hovering over an `Operation` may show its generic format. 114 115 116 117#### Navigation 118 119The language server will also inform the editor about the structure of symbol 120tables within the IR. This allows for jumping directly to the definition of a 121symbol, such as a `func.func`, within the file. 122 123 124 125#### Bytecode Editing and Inspection 126 127The language server provides support for interacting with MLIR bytecode files, 128enabling IDEs to transparently view and edit bytecode files in the same way 129as textual `.mlir` files. 130 131 132 133## PDLL LSP Language Server : `mlir-pdll-lsp-server` 134 135MLIR provides an implementation of an LSP language server for `.pdll` text files 136in the form of the `mlir-pdll-lsp-server` tool. This tool interacts with the 137PDLL C++ API to support rich language queries, such as code completion and "Find 138Definition". 139 140### Compilation Database 141 142Similarly to 143[`clangd`](https://clang.llvm.org/docs/JSONCompilationDatabase.html), and 144language servers for various other programming languages, the PDLL language 145server relies on a compilation database to provide build-system information for 146`.pdll` files. This information includes, for example, the include directories 147available for that file. This database allows for the server to interact with 148`.pdll` files using the same configuration as when building. 149 150#### Format 151 152A PDLL compilation database is a YAML file, conventionally named 153`pdll_compile_commands.yml`, that contains a set of `FileInfo` documents 154providing information for individiual `.pdll` files. 155 156Example: 157 158```yaml 159--- !FileInfo: 160 filepath: "/home/user/llvm/mlir/lib/Dialect/Arith/IR/ArithCanonicalization.pdll" 161 includes: "/home/user/llvm/mlir/lib/Dialect/Arith/IR;/home/user/llvm/mlir/include" 162``` 163 164- filepath: <string> - Absolute file path of the file. 165- includes: <string> - Semi-colon delimited list of absolute include 166 directories. 167 168#### Build System Integration 169 170Per convention, PDLL compilation databases should be named 171`pdll_compile_commands.yml` and placed at the top of the build directory. When 172using CMake and `mlir_pdll`, a compilation database is generally automatically 173built and placed in the appropriate location. 174 175### Features 176 177This section details a few of the features that the PDLL language server 178provides. The screenshots are shown in [VSCode](https://code.visualstudio.com/), 179but the exact feature set available will depend on your editor client. 180 181[pdll features]: # 182 183#### Diagnostics 184 185The language server actively runs verification as you type, showing any 186generated diagnostics in-place. 187 188 189 190#### Code completion and signature help 191 192The language server provides suggestions as you type based on what constraints, 193rewrites, dialects, operations, etc are available in this context. The server 194also provides information about the structure of constraint and rewrite calls, 195operations, and more as you fill them in. 196 197 198 199#### Cross-references 200 201Cross references allow for navigating the code base. 202 203##### Find definition 204 205Jump to the definition of a symbol under the cursor: 206 207 208 209If ODS information is available, we can also jump to the definition of operation 210names and more: 211 212 213 214##### Find references 215 216Show all references of the symbol under the cursor. 217 218 219 220#### Hover 221 222Hover over a symbol to see more information about it, such as its type, 223documentation, and more. 224 225 226 227If ODS information is available, we can also show information directly from the 228operation definitions: 229 230 231 232#### Navigation 233 234The language server will also inform the editor about the structure of symbols 235within the IR. 236 237 238 239#### View intermediate output 240 241The language server provides support for introspecting various intermediate 242stages of compilation, such as the AST, the `.mlir` containing the generated 243PDL, and the generated C++ glue. This is a custom LSP extension, and is not 244necessarily provided by all IDE clients. 245 246 247 248#### Inlay hints 249 250The language server provides additional information inline with the source code. 251Editors usually render this using read-only virtual text snippets interspersed 252with code. Hints may be shown for: 253 254- types of local variables 255- names of operand and result groups 256- constraint and rewrite arguments 257 258 259 260## TableGen LSP Language Server : `tblgen-lsp-server` 261 262MLIR provides an implementation of an LSP language server for `.td` text files 263in the form of the `tblgen-lsp-server` tool. This tool interacts with the 264TableGen C++ API to support rich language queries, such as "Find Definition". 265 266### Compilation Database 267 268Similarly to 269[`clangd`](https://clang.llvm.org/docs/JSONCompilationDatabase.html), and 270language servers for various other programming languages, the TableGen language 271server relies on a compilation database to provide build-system information for 272`.td` files. This information includes, for example, the include directories 273available for that file. This database allows for the server to interact with 274`.td` files using the same configuration as when building. 275 276#### Format 277 278A TableGen compilation database is a YAML file, conventionally named 279`tablegen_compile_commands.yml`, that contains a set of `FileInfo` documents 280providing information for individiual `.td` files. 281 282Example: 283 284```yaml 285--- !FileInfo: 286 filepath: "/home/user/llvm/mlir/lib/Dialect/Arith/IR/ArithCanonicalization.td" 287 includes: "/home/user/llvm/mlir/lib/Dialect/Arith/IR;/home/user/llvm/mlir/include" 288``` 289 290- filepath: <string> - Absolute file path of the file. 291- includes: <string> - Semi-colon delimited list of absolute include 292 directories. 293 294#### Build System Integration 295 296Per convention, TableGen compilation databases should be named 297`tablegen_compile_commands.yml` and placed at the top of the build directory. 298When using CMake and `mlir_tablegen`, a compilation database is generally 299automatically built and placed in the appropriate location. 300 301### Features 302 303This section details a few of the features that the TableGen language server 304provides. The screenshots are shown in [VSCode](https://code.visualstudio.com/), 305but the exact feature set available will depend on your editor client. 306 307[tablegen features]: # 308 309#### Diagnostics 310 311The language server actively runs verification as you type, showing any 312generated diagnostics in-place. 313 314 315 316#### Cross-references 317 318Cross references allow for navigating the code base. 319 320##### Find definition 321 322Jump to the definition of a symbol under the cursor: 323 324 325 326##### Find references 327 328Show all references of the symbol under the cursor. 329 330 331 332#### Hover 333 334Hover over a symbol to see more information about it, such as its type, 335documentation, and more. 336 337 338 339Hovering over an overridden field will also show you information such as 340documentation from the base value: 341 342 343 344## Language Server Design 345 346The design of the various language servers provided by MLIR are effectively the 347same, and are largely comprised of three different components: 348 349- Communication and Transport (via JSON-RPC) 350- Language Server Protocol 351- Language-Specific Server 352 353 354 355### Communication and Transport 356 357The language server, such as `mlir-lsp-server`, communicates with the language 358client via JSON-RPC over stdin/stdout. In the code, this is the `JSONTransport` 359class. This class knows nothing about the Language Server Protocol, it only 360knows that JSON-RPC messages are coming in and JSON-RPC messages are going out. 361The handling of incoming and outgoing LSP messages is left to the 362`MessageHandler` class. This class routes incoming messages to handlers in the 363`Language Server Protocol` layer for interpretation, and packages outgoing 364messages for transport. This class also has limited knowledge of the LSP, and 365only has information about the three main classes of messages: notifications, 366calls, and replies. 367 368### Language Server Protocol 369 370`LSPServer` handles the interpretation of the finer LSP details. This class 371registers handlers for LSP messages and then forwards to the 372[`Language-Specific Server`](#language-specific-server) for processing. The 373intent of this component is to hold all of the necessary glue when communicating 374from the LSP world to the language-specific world (e.g. MLIR, PDLL, etc.). In 375most cases, the LSP message handlers simply forward directly to the 376`Language-Specific Server`. In some cases, however, the impedance mismatch 377between the two requires more complicated glue code. 378 379### Language-Specific Server 380 381The language specific server, such as `MLIRServer` or `PDLLServer`, provides the 382internal implementation of all of LSP queries for a specific language. These are 383the classes that directly interacts with the C++ API for the language, including 384parsing text files, interpreting definition/reference information, etc. 385 386## Editor Plugins 387 388LSP Language plugins are available for many popular editors, and in principle 389the language servers provided by MLIR should work with any of them, though 390feature sets and interfaces may vary. Below are a set of plugins that are known 391to work: 392 393### Visual Studio Code 394 395The [MLIR extension](https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-mlir) 396provides language IDE features for [MLIR](https://mlir.llvm.org/) related 397languages: [MLIR](#mlir---mlir-textual-assembly-format), 398[PDLL](#pdll---mlir-pdll-pattern-files), and [TableGen](#td---tablegen-files) 399 400#### `.mlir` - MLIR textual assembly format: 401 402The MLIR extension adds language support for the 403[MLIR textual assembly format](https://mlir.llvm.org/docs/LangRef/): 404 405##### Features 406 407- Syntax highlighting for `.mlir` files and `mlir` markdown blocks 408- go-to-definition and cross references 409- Detailed information when hovering over IR entities 410- Outline and navigation of symbols and symbol tables 411- Code completion 412- Live parser and verifier diagnostics 413 414[mlir-vscode features]: # 415 416##### Setup 417 418###### `mlir-lsp-server` 419 420The various `.mlir` language features require the 421[`mlir-lsp-server` language server](https://mlir.llvm.org/docs/Tools/MLIRLSP/#mlir-lsp-language-server--mlir-lsp-server). 422If `mlir-lsp-server` is not found within your workspace path, you must specify 423the path of the server via the `mlir.server_path` setting. The path of the 424server may be absolute or relative within your workspace. 425 426#### `.pdll` - MLIR PDLL pattern files: 427 428The MLIR extension adds language support for the 429[PDLL pattern language](https://mlir.llvm.org/docs/PDLL/). 430 431##### Features 432 433- Syntax highlighting for `.pdll` files and `pdll` markdown blocks 434- go-to-definition and cross references 435- Types and documentation on hover 436- Code completion and signature help 437- View intermediate AST, MLIR, or C++ output 438 439[pdll-vscode features]: # 440 441##### Setup 442 443###### `mlir-pdll-lsp-server` 444 445The various `.pdll` language features require the 446[`mlir-pdll-lsp-server` language server](https://mlir.llvm.org/docs/Tools/MLIRLSP/#pdll-lsp-language-server--mlir-pdll-lsp-server). 447If `mlir-pdll-lsp-server` is not found within your workspace path, you must 448specify the path of the server via the `mlir.pdll_server_path` setting. The path 449of the server may be absolute or relative within your workspace. 450 451###### Project setup 452 453To properly understand and interact with `.pdll` files, the language server must 454understand how the project is built (compile flags). 455[`pdll_compile_commands.yml` files](https://mlir.llvm.org/docs/Tools/MLIRLSP/#compilation-database) 456related to your project should be provided to ensure files are properly 457processed. These files can usually be generated by the build system, and the 458server will attempt to find them within your `build/` directory. If not 459available in or a unique location, additional `pdll_compile_commands.yml` files 460may be specified via the `mlir.pdll_compilation_databases` setting. The paths of 461these databases may be absolute or relative within your workspace. 462 463#### `.td` - TableGen files: 464 465The MLIR extension adds language support for the 466[TableGen language](https://llvm.org/docs/TableGen/ProgRef.html). 467 468##### Features 469 470- Syntax highlighting for `.td` files and `tablegen` markdown blocks 471- go-to-definition and cross references 472- Types and documentation on hover 473 474[tablegen-vscode features]: # 475 476##### Setup 477 478###### `tblgen-lsp-server` 479 480The various `.td` language features require the 481[`tblgen-lsp-server` language server](https://mlir.llvm.org/docs/Tools/MLIRLSP/#tablegen-lsp-language-server--tblgen-lsp-server). 482If `tblgen-lsp-server` is not found within your workspace path, you must specify 483the path of the server via the `mlir.tablegen_server_path` setting. The path of 484the server may be absolute or relative within your workspace. 485 486###### Project setup 487 488To properly understand and interact with `.td` files, the language server must 489understand how the project is built (compile flags). 490[`tablegen_compile_commands.yml` files](https://mlir.llvm.org/docs/Tools/MLIRLSP/#compilation-database-1) 491related to your project should be provided to ensure files are properly 492processed. These files can usually be generated by the build system, and the 493server will attempt to find them within your `build/` directory. If not 494available in or a unique location, additional `tablegen_compile_commands.yml` 495files may be specified via the `mlir.tablegen_compilation_databases` setting. 496The paths of these databases may be absolute or relative within your workspace. 497 498#### Contributing 499 500This extension is actively developed within the 501[LLVM monorepo](https://github.com/llvm/llvm-project), at 502[`mlir/utils/vscode`](https://github.com/llvm/llvm-project/tree/main/mlir/utils/vscode). 503As such, contributions should follow the 504[normal LLVM guidelines](https://llvm.org/docs/Contributing.html), with code 505reviews sent to 506[GitHub](https://llvm.org/docs/Contributing.html#how-to-submit-a-patch). 507 508When developing or deploying this extension within the LLVM monorepo, a few 509extra setup steps are required: 510 511- Copy `mlir/utils/textmate/mlir.json` to the extension directory and rename to 512 `grammar.json`. 513- Copy `llvm/utils/textmate/tablegen.json` to the extension directory and rename 514 to `tablegen-grammar.json`. 515- Copy 516 `https://mlir.llvm.org//LogoAssets/logo/PNG/full_color/mlir-identity-03.png` 517 to the extension directory and rename to `icon.png`. 518 519Please follow the existing code style when contributing to the extension, we 520recommend to run `npm run format` before sending a patch. 521
