Lines Matching full:the

3 This document describes the MLIR bytecode format and its encoding.
7 older version of the format can be targetted.
9 That said, it is important to realize that the promises of the
10 bytecode format are made assuming immutable dialects: the format
14 A dialect can opt-in to handle its own versioning through the
15 `BytecodeDialectInterface`. Some hooks are exposed to the dialect
16 to allow managing a version encoded into the bytecode file. The
17 version is loaded lazily and allows to retrieve the version
18 information while decoding the input IR, and gives an opportunity
20 upgrades post-parsing through the `upgradeFromVersion` method.
28 MLIR uses the following four-byte magic number to
50 Fixed width integers are unsigned integers of a known byte size. The values are
59 represent a single 64-bit value. The MLIR bytecode utilizes the "PrefixVarInt"
60 encoding for VarInts. This encoding is a variant of the
62 encoding, where each byte of the encoding provides up to 7 bits for the value,
63 with the remaining bit used to store a tag indicating the number of bytes used
64 for the encoding. This means that small unsigned integers (less than 2^7) may be
68 The first byte of the encoding includes a length prefix in the low bits. This
69 prefix is a bit sequence of '0's followed by a terminal '1', or the end of the
70 byte. The number of '0' bits indicate the number of _additional_ bytes, not
71 including the prefix byte, used to encode the value. All of the remaining bits
72 in the first byte, along with all of the bits in the additional bytes, provide
73 the value of the integer. Below are the various possible encodings of the prefix
77 xxxxxxx1:  7 value bits, the encoding uses 1 byte
78 xxxxxx10: 14 value bits, the encoding uses 2 bytes
79 xxxxx100: 21 value bits, the encoding uses 3 bytes
80 xxxx1000: 28 value bits, the encoding uses 4 bytes
81 xxx10000: 35 value bits, the encoding uses 5 bytes
82 xx100000: 42 value bits, the encoding uses 6 bytes
83 x1000000: 49 value bits, the encoding uses 7 bytes
84 10000000: 56 value bits, the encoding uses 8 bytes
85 00000000: 64 value bits, the encoding uses 9 bytes
93 This encoding uses the low bit of the value to indicate the sign, which allows
97 number of active bits in the value, leading to a smaller encoding. Below is the
122 Sections are a mechanism for grouping data within the bytecode. They enable
125 indicates if the section has alignment requirements, a length (which allows for
126 skipping over the section), and an optional alignment. When an alignment is
127 present, a variable number of padding bytes (0xCB) may appear before the section
128 data. The alignment of a section must be a power of 2.
132 Given the generic structure of MLIR, the bytecode encoding is actually fairly
133 simplistic. It effectively maps to the core components of MLIR.
137 The top-level structure of the bytecode contains the 4-byte "magic number", a
160 The string section contains a table of strings referenced within the bytecode,
161 more easily enabling string sharing. This section is encoded first with the
162 total number of strings, followed by the sizes of each of the individual strings
163 in reverse order. The remaining encoding contains a single blob containing all
164 of the strings concatenated together.
168 The dialect section of the bytecode contains all of the dialects referenced
169 within the encoded IR, and some information about the components of those
200 Dialects are encoded as a `varint` containing the index to the name string
201 within the string section, plus a flag indicating whether the dialect is
203 containing the dialect, the number of operation names, and the array of indexes
204 to each name within the string section. The version is encoded as a nested
210 (`attr_type_section`) containing the actual encoded representation, and another
211 section (`attr_type_offset_section`) containing the offsets of each encoded
212 attribute/type into the previous section. This structure allows for attributes
240 Each `offset` in the `attr_type_offset_section` above is the size of the
241 encoding for the attribute or type and a flag indicating if the encoding uses
242 the textual assembly format, or a custom bytecode encoding. We avoid using the
243 direct offset into the `attr_type_section`, as a smaller relative offsets
245 dialect, with each `attr_type_offset_group` in the offset section containing the
247 within the group.
251 In the abstract, an attribute/type is encoded in one of two possible ways: via
256 In the case where a dialect does not define a method for encoding the attribute
257 or type, the textual assembly format of that attribute or type is used as a
258 fallback. For example, a type `!bytecode.type<42>` would be encoded as the null
260 type can be encoded, even if the owning dialect has not yet opted in to a more
263 TODO: We shouldn't redundantly encode the dialect name here, we should use a
264 reference to the parent dialect instead.
268 As an alternative to the assembly format fallback, dialects may also provide a
272 Dialects can opt-in to providing custom encodings by implementing the
275 by the bytecode reader and writer. These hooks are provided a reader and writer
276 implementation that can be used to encode various constructs in the underlying
282 When implementing the bytecode interface, dialects are responsible for all
283 aspects of the encoding. This includes the indicator for which kind of attribute
284 or type is being encoded; the bytecode reader will only know that it has
287 `varint` code to indicate how the attribute or type was encoded.
292 (`resource_section`) containing the actual encoded representation, and another
293 section (`resource_offset_section`) containing the offsets of each encoded
294 resource into the previous section.
332 Resources are grouped by the provider, either an external entity or a dialect,
333 with each `resource_group` in the offset section containing the corresponding
334 provider, number of elements, and info for each element within the group. For
335 each element, we record the key, the value kind, and the encoded size. We avoid
336 using the direct offset into the `resource_section`, as a smaller relative
341 The IR section contains the encoded form of operations within the bytecode.
384 The encoding of an operation is important because this is generally the most
385 commonly appearing structure in the bytecode. A single encoding is used for
386 every type of operation. Given this prevalence, many of the fields of an
387 operation are optional. The `encodingMask` field is a bitmask which indicates
388 which of the components of the operation are present.
392 The location is encoded as the index of the location within the attribute table.
396 If the operation has attribues, the index of the operation attribute dictionary
397 within the attribute table is encoded.
401 If the operation has results, the number of results and the indexes of the
402 result types within the type table are encoded.
406 If the operation has operands, the number of operands and the value index of
407 each operand is encoded. This value index is the relative ordering of the
408 definition of that value from the start of the first ancestor isolated region.
412 If the operation has successors, the number of successors and the indexes of the
413 successor blocks within the parent region are encoded.
417 The reference use-list order is assumed to be the reverse of the global
418 enumeration of all the op operands that one would obtain with a pre-order walk
419 of the IR. This order is naturally obtained by building blocks of operations
420 op-by-op. However, some transformations may shuffle the use-lists with respect
421 to this reference ordering. If any of the results of the operation have a
422 use-list order that is not sorted with respect to the reference use-list order,
424 parsing the bytecode. The encoding represents an index map from the reference
425 operand order to the current use-list order. A bit flag is used to detect if
426 this encoding is of type index-pair or not. When the bit flag is set to zero,
427 the element at `i` represent the position of the use `i` of the reference list
428 into the current use-list. When the bit flag is set to `1`, the encoding
429 represent index pairs `(i, j)`, which indicate that the use at position `i` of
430 the reference list is mapped to position `j` in the current use-list. When only
431 less than half of the elements in the current use-list are shuffled with respect
432 to the reference use-list, the index-pair encoding is used to reduce the
437 If the operation has regions, the number of regions and if the regions are
452 A region is encoded first with the number of blocks within. If the region is
453 non-empty, the number of values defined directly within the region are encoded,
454 followed by the blocks of the region.
478 A block is encoded with an array of operations and block arguments. The first
479 field is an encoding that combines the number of operations in the block, with a
480 flag indicating if the block has arguments.