1.. _header_generation: 2 3Generating Public and Internal headers 4====================================== 5 6There are 3 main components of the Headergen. The first component are the YAML 7files that contain all the function header information and are separated by 8header specification and standard. The second component are the classes that 9are created for each component of the function header: macros, enumerations, 10types, function, arguments, and objects. The third component is the Python 11script that uses the class representation to deserialize YAML files into its 12specific components and then reserializes the components into the function 13header. The Python script also combines the generated header content with 14header definitions and extra macro and type inclusions from the .h.def file. 15 16 17Instructions 18------------ 19 20Required Versions: 21 - Python Version: 3.8 22 - PyYAML Version: 5.1 23 241. Keep full-build mode on when building, otherwise headers will not be 25 generated. 262. Once the build is complete, enter in the command line within the build 27 directory ``ninja check-hdrgen`` to ensure that the integration tests are 28 passing. 293. Then enter in the command line ``ninja libc`` to generate headers. Headers 30 will be in ``build/projects/libc/include`` or ``build/libc/include`` in a 31 runtime build. Sys spec headers will be located in 32 ``build/projects/libc/include/sys``. 33 34 35To add a function to the YAML files, you can either manually enter it in the 36YAML file corresponding to the header it belongs to or add it through the 37command line. 38 39To add through the command line: 40 411. Make sure you are in the llvm-project directory. 42 432. Enter in the command line: 44 45 .. code-block:: none 46 47 python3 libc/utils/hdrgen/yaml_to_classes.py 48 libc/include/[yaml_file.yaml] --add_function "<return_type>" <function_name> "<function_arg1, function_arg2>" <standard> <guard> <attribute> 49 50 Example: 51 52 .. code-block:: none 53 54 python3 libc/utils/hdrgen/yaml_to_classes.py 55 libc/include/ctype.yaml --add_function "char" example_function 56 "int, void, const void" stdc example_float example_attribute 57 58 Keep in mind only the return_type and arguments have quotes around them. If 59 you do not have any guards or attributes you may enter "null" for both. 60 613. Check the YAML file that the added function is present. You will also get a 62 generated header file with the new addition in the hdrgen directory to 63 examine. 64 65If you want to sort the functions alphabetically you can check out 66libc/utils/hdrgen/yaml_functions_sorted.py. 67 68 69Testing 70------- 71 72Headergen has an integration test that you may run once you have configured 73your CMake within the build directory. In the command line, enter the 74following: ``ninja check-hdrgen``. The integration test is one test that 75ensures the process of YAML to classes to generate headers works properly. If 76there are any new additions on formatting headers, make sure the test is 77updated with the specific addition. 78 79Integration Test can be found in: ``libc/utils/hdrgen/tests/test_integration.py`` 80 81File to modify if adding something to formatting: 82``libc/utils/hdrgen/tests/expected_output/test_header.h`` 83 84 85Common Errors 86------------- 871. Missing function specific component 88 89 Example: 90 91 .. code-block:: none 92 93 "/llvm-project/libc/utils/hdrgen/yaml_to_classes.py", line 67, in yaml_to_classes function_data["return_type"] 94 95 If you receive this error or any error pertaining to 96 ``function_data[function_specific_component]`` while building the headers 97 that means the function specific component is missing within the YAML files. 98 Through the call stack, you will be able to find the header file which has 99 the issue. Ensure there is no missing function specific component for that 100 YAML header file. 101 1022. CMake Error: require argument to be specified 103 104 Example: 105 106 .. code-block:: none 107 108 CMake Error at: 109 /llvm-project/libc/cmake/modules/LLVMLibCHeaderRules.cmake:86 (message): 110 'add_gen_hdr2' rule requires GEN_HDR to be specified. 111 Call Stack (most recent call first): 112 /llvm-project/libc/include/CMakeLists.txt:22 (add_gen_header2) 113 /llvm-project/libc/include/CMakeLists.txt:62 (add_header_macro) 114 115 If you receive this error, there is a missing YAML file, h_def file, or 116 header name within the ``libc/include/CMakeLists.txt``. The last line in the 117 error call stack will point to the header where there is a specific component 118 missing. Ensure the correct style and required files are present: 119 120 | ``[header_name]`` 121 | ``[../libc/include/[yaml_file.yaml]`` 122 | ``[header_name.h.def]`` 123 | ``[header_name.h]`` 124 | ``DEPENDS`` 125 | ``{Necessary Depend Files}`` 126 1273. Command line: expected arguments 128 129 Example: 130 131 .. code-block:: none 132 133 usage: yaml_to_classes.py [-h] [--output_dir OUTPUT_DIR] [--h_def_file H_DEF_FILE] 134 [--add_function RETURN_TYPE NAME ARGUMENTS STANDARDS GUARD ATTRIBUTES] 135 [--e ENTRY_POINTS] [--export-decls] 136 yaml_file 137 yaml_to_classes.py: 138 error: argument --add_function: expected 6 arguments 139 140 In the process of adding a function, you may run into an issue where the 141 command line is requiring more arguments than what you currently have. Ensure 142 that all components of the new function are filled. Even if you do not have a 143 guard or attribute, make sure to put null in those two areas. 144 1454. Object has no attribute 146 147 Example: 148 149 .. code-block:: none 150 151 File "/llvm-project/libc/utils/hdrgen/header.py", line 60, in __str__ for 152 function in self.functions: AttributeError: 'HeaderFile' object has no 153 attribute 'functions' 154 155 When running ``ninja libc`` in the build directory to generate headers you 156 may receive the error above. Essentially this means that in 157 ``libc/utils/hdrgen/header.py`` there is a missing attribute named functions. 158 Make sure all function components are defined within this file and there are 159 no missing functions to add these components. 160 1615. Unknown type name 162 163 Example: 164 165 .. code-block:: none 166 167 /llvm-project/build/projects/libc/include/sched.h:20:25: error: unknown type 168 name 'size_t'; did you mean 'time_t'? 169 20 | int_sched_getcpucount(size_t, const cpu_set_t*) __NOEXCEPT 170 | ^ 171 /llvm-project/build/projects/libc/include/llvm-libc-types/time_t.h:15:24: 172 note: 'time_t' declared here 173 15 | typedef __INT64_TYPE__ time_t; 174 | ^ 175 176 During the header generation process errors like the one above may occur 177 because there are missing types for a specific header file. Check the YAML 178 file corresponding to the header file and make sure all the necessary types 179 that are being used are input into the types as well. Delete the specific 180 header file from the build folder and re-run ``ninja libc`` to ensure the 181 types are being recognized. 182 1836. Test Integration Errors 184 185 Sometimes the integration test will fail but that 186 still means the process is working unless the comparison between the output 187 and expected_output is not showing. If that is the case make sure in 188 ``libc/utils/hdrgen/tests/test_integration.py`` there are no missing arguments 189 that run through the script. 190 191 If the integration tests are failing due to mismatching of lines or small 192 errors in spacing that is nothing to worry about. If this is happening while 193 you are making a new change to the formatting of the headers, then 194 ensure the expected output file 195 ``libc/utils/hdrgen/tests/expected_output/test_header.h`` has the changes you 196 are applying. 197