xref: /llvm-project/libc/docs/dev/header_generation.rst (revision 62cd050b635cbb201dd08188696448cf5ab23260)
1.. _header_generation:
2
3Generating Public and Internal headers
4======================================
5
6There are 3 main components of the Headergen. The first component are the YAML
7files that contain all the function header information and are separated by
8header specification and standard. The second component are the classes that
9are created for each component of the function header: macros, enumerations,
10types, function, arguments, and objects. The third component is the Python
11script that uses the class representation to deserialize YAML files into its
12specific components and then reserializes the components into the function
13header. The Python script also combines the generated header content with
14header definitions and extra macro and type inclusions from the .h.def file.
15
16
17Instructions
18------------
19
20Required Versions:
21  - Python Version: 3.8
22  - PyYAML Version: 5.1
23
241. Keep full-build mode on when building, otherwise headers will not be
25   generated.
262. Once the build is complete, enter in the command line within the build
27   directory ``ninja check-hdrgen`` to ensure that the integration tests are
28   passing.
293. Then enter in the command line ``ninja libc`` to generate headers. Headers
30   will be in ``build/projects/libc/include`` or ``build/libc/include`` in a
31   runtime build. Sys spec headers will be located in
32   ``build/projects/libc/include/sys``.
33
34
35To add a function to the YAML files, you can either manually enter it in the
36YAML file corresponding to the header it belongs to or add it through the
37command line.
38
39To add through the command line:
40
411. Make sure you are in the llvm-project directory.
42
432. Enter in the command line:
44
45   .. code-block:: none
46
47     python3 libc/utils/hdrgen/yaml_to_classes.py
48     libc/include/[yaml_file.yaml] --add_function "<return_type>" <function_name> "<function_arg1, function_arg2>" <standard> <guard> <attribute>
49
50   Example:
51
52   .. code-block:: none
53
54      python3 libc/utils/hdrgen/yaml_to_classes.py
55      libc/include/ctype.yaml --add_function "char" example_function
56      "int, void, const void" stdc example_float example_attribute
57
58   Keep in mind only the return_type and arguments have quotes around them. If
59   you do not have any guards or attributes you may enter "null" for both.
60
613. Check the YAML file that the added function is present. You will also get a
62   generated header file with the new addition in the hdrgen directory to
63   examine.
64
65If you want to sort the functions alphabetically you can check out
66libc/utils/hdrgen/yaml_functions_sorted.py.
67
68
69Testing
70-------
71
72Headergen has an integration test that you may run once you have configured
73your CMake within the build directory. In the command line, enter the
74following: ``ninja check-hdrgen``. The integration test is one test that
75ensures the process of YAML to classes to generate headers works properly. If
76there are any new additions on formatting headers, make sure the test is
77updated with the specific addition.
78
79Integration Test can be found in: ``libc/utils/hdrgen/tests/test_integration.py``
80
81File to modify if adding something to formatting:
82``libc/utils/hdrgen/tests/expected_output/test_header.h``
83
84
85Common Errors
86-------------
871. Missing function specific component
88
89   Example:
90
91   .. code-block:: none
92
93      "/llvm-project/libc/utils/hdrgen/yaml_to_classes.py", line 67, in yaml_to_classes function_data["return_type"]
94
95   If you receive this error or any error pertaining to
96   ``function_data[function_specific_component]`` while building the headers
97   that means the function specific component is missing within the YAML files.
98   Through the call stack, you will be able to find the header file which has
99   the issue. Ensure there is no missing function specific component for that
100   YAML header file.
101
1022. CMake Error: require argument to be specified
103
104   Example:
105
106   .. code-block:: none
107
108     CMake Error at:
109     /llvm-project/libc/cmake/modules/LLVMLibCHeaderRules.cmake:86 (message):
110     'add_gen_hdr2' rule requires GEN_HDR to be specified.
111     Call Stack (most recent call first):
112     /llvm-project/libc/include/CMakeLists.txt:22 (add_gen_header2)
113     /llvm-project/libc/include/CMakeLists.txt:62 (add_header_macro)
114
115   If you receive this error, there is a missing YAML file, h_def file, or
116   header name within the ``libc/include/CMakeLists.txt``. The last line in the
117   error call stack will point to the header where there is a specific component
118   missing. Ensure the correct style and required files are present:
119
120   | ``[header_name]``
121   | ``[../libc/include/[yaml_file.yaml]``
122   | ``[header_name.h.def]``
123   | ``[header_name.h]``
124   | ``DEPENDS``
125   |   ``{Necessary Depend Files}``
126
1273. Command line: expected arguments
128
129   Example:
130
131   .. code-block:: none
132
133     usage: yaml_to_classes.py [-h] [--output_dir OUTPUT_DIR] [--h_def_file H_DEF_FILE]
134     [--add_function RETURN_TYPE NAME ARGUMENTS STANDARDS GUARD ATTRIBUTES]
135     [--e ENTRY_POINTS] [--export-decls]
136     yaml_file
137     yaml_to_classes.py:
138     error: argument --add_function: expected 6 arguments
139
140   In the process of adding a function, you may run into an issue where the
141   command line is requiring more arguments than what you currently have. Ensure
142   that all components of the new function are filled. Even if you do not have a
143   guard or attribute, make sure to put null in those two areas.
144
1454. Object has no attribute
146
147   Example:
148
149   .. code-block:: none
150
151     File "/llvm-project/libc/utils/hdrgen/header.py", line 60, in __str__ for
152     function in self.functions: AttributeError: 'HeaderFile' object has no
153     attribute 'functions'
154
155   When running ``ninja libc`` in the build directory to generate headers you
156   may receive the error above. Essentially this means that in
157   ``libc/utils/hdrgen/header.py`` there is a missing attribute named functions.
158   Make sure all function components are defined within this file and there are
159   no missing functions to add these components.
160
1615. Unknown type name
162
163   Example:
164
165   .. code-block:: none
166
167     /llvm-project/build/projects/libc/include/sched.h:20:25: error: unknown type
168     name 'size_t'; did you mean 'time_t'?
169     20 | int_sched_getcpucount(size_t, const cpu_set_t*) __NOEXCEPT
170      |           ^
171     /llvm-project/build/projects/libc/include/llvm-libc-types/time_t.h:15:24:
172     note: 'time_t' declared here
173     15 | typedef __INT64_TYPE__ time_t;
174     |                    ^
175
176   During the header generation process errors like the one above may occur
177   because there are missing types for a specific header file. Check the YAML
178   file corresponding to the header file and make sure all the necessary types
179   that are being used are input into the types as well. Delete the specific
180   header file from the build folder and re-run ``ninja libc`` to ensure the
181   types are being recognized.
182
1836. Test Integration Errors
184
185   Sometimes the integration test will fail but that
186   still means the process is working unless the comparison between the output
187   and expected_output is not showing. If that is the case make sure in
188   ``libc/utils/hdrgen/tests/test_integration.py`` there are no missing arguments
189   that run through the script.
190
191   If the integration tests are failing due to mismatching of lines or small
192   errors in spacing that is nothing to worry about. If this is happening while
193   you are making a new change to the formatting of the headers, then
194   ensure the expected output file
195   ``libc/utils/hdrgen/tests/expected_output/test_header.h`` has the changes you
196   are applying.
197