| #
072e81db |
| 02-Jul-2024 |
Aaron Ballman <aaron@aaronballman.com> |
Revert "[Clang][Comments] Attach comments to decl even if preproc directives are in between (#88367)"
This reverts commit 9f04d75b2bd8ba83863db74ebe1a5c08cfc5815c.
There was post-commit feedback on
Revert "[Clang][Comments] Attach comments to decl even if preproc directives are in between (#88367)"
This reverts commit 9f04d75b2bd8ba83863db74ebe1a5c08cfc5815c.
There was post-commit feedback on the direction this PR took.
show more ...
|
| #
9f04d75b |
| 01-Jul-2024 |
hdoc <68132204+hdoc@users.noreply.github.com> |
[Clang][Comments] Attach comments to decl even if preproc directives are in between (#88367)
### Background
It's surprisingly common for C++ code in the wild to conditionally
show/hide declarati
[Clang][Comments] Attach comments to decl even if preproc directives are in between (#88367)
### Background
It's surprisingly common for C++ code in the wild to conditionally
show/hide declarations to Doxygen through the use of preprocessor
directives. One especially common version of this pattern is
demonstrated below:
```cpp
/// @brief Test comment
#ifdef DOXYGEN_BUILD_ENABLED
template<typename T>
#else
template <typename T>
typename std::enable_if<std::is_integral<T>::value>::type
#endif
void f() {}
```
There are more examples I've collected below to demonstrate usage of
this pattern:
- Example 1:
[Magnum](https://github.com/mosra/magnum/blob/8538610fa27e1db37070eaabe34f1e4e41648bab/src/Magnum/Resource.h#L117-L127)
- Example 2:
[libcds](https://github.com/khizmax/libcds/blob/9985d2a87feaa3e92532e28f4ab762a82855a49c/cds/container/michael_list_nogc.h#L36-L54)
- Example 3:
[rocPRIM](https://github.com/ROCm/rocPRIM/blob/609ae19565ff6a3499168b76a0be5652762e24f6/rocprim/include/rocprim/block/detail/block_reduce_raking_reduce.hpp#L60-L65)
From my research, it seems like the most common rationale for this
functionality is hiding difficult-to-parse code from Doxygen, especially
where template metaprogramming is concerned.
Currently, Clang does not support attaching comments to decls if there
are preprocessor comments between the comment and the decl. This is
enforced here:
https://github.com/llvm/llvm-project/blob/b6ebea7972cd05a8e4dcf0d5a54f2c793999995a/clang/lib/AST/ASTContext.cpp#L284-L287
Alongside preprocessor directives, any instance of `;{}#@` between a
comment and decl will cause the comment to not be attached to the decl.
#### Rationale
It would be nice for Clang-based documentation tools, such as
[hdoc](https://hdoc.io), to support code using this pattern. Users
expect to see comments attached to the relevant decl — even if there is
an `#ifdef` in the way — which Clang does not currently do.
#### History
Originally, commas were also in the list of "banned" characters, but
were removed in `b534d3a0ef69`
([link](https://github.com/llvm/llvm-project/commit/b534d3a0ef6970f5e42f10ba5cfcb562d8b184e1))
because availability macros often have commas in them. From my reading
of the code, it appears that the original intent of the code was to
exclude macros and decorators between comments and decls, possibly in an
attempt to properly attribute comments to macros (discussed further in
"Complications", below). There's some more discussion here:
https://reviews.llvm.org/D125061.
### Change
This modifies Clang comment parsing so that comments are attached to
subsequent declarations even if there are preprocessor directives
between the end of the comment and the start of the decl. Furthermore,
this change:
- Adds tests to verify that comments are attached to their associated
decls even if there are preprocessor directives in between
- Adds tests to verify that current behavior has not changed (i.e. use
of the other characters between comment and decl will result in the
comment not being attached to the decl)
- Updates existing `lit` tests which would otherwise break.
#### Complications
Clang [does not yet
support](https://github.com/llvm/llvm-project/issues/38206) attaching
doc comments to macros. Consequently, the change proposed in this RFC
affects cases where a doc comment attached to a macro is followed
immediately by a normal declaration. In these cases, the macro's doc
comments will be attached to the subsequent decl. Previously they would
be ignored because any preprocessor directives between a comment and a
decl would result in the comment not being attached to the decl. An
example of this is shown below.
```cpp
/// Doc comment for a function-like macro
/// @param n
/// A macro argument
#define custom_sqrt(n) __internal_sqrt(n)
int __internal_sqrt(int n) { return __builtin_sqrt(n); }
// NB: the doc comment for the custom_sqrt macro will actually be attached to __internal_sqrt!
```
There is a real instance of this problem in the Clang codebase, namely
here:
https://github.com/llvm/llvm-project/blob/be10070f91b86a6f126d2451852242bfcb2cd366/clang/lib/Headers/amxcomplexintrin.h#L65-L114
As part of this RFC, I've added a semicolon to break up the Clang
comment parsing so that the `-Wdocumentation` errors go away, but this
is a hack. The real solution is to fix Clang comment parsing so that doc
comments are properly attached to macros, however this would be a large
change that is outside of the scope of this RFC.
show more ...
|
|
Revision tags: llvmorg-18.1.8, llvmorg-18.1.7, llvmorg-18.1.6, llvmorg-18.1.5, llvmorg-18.1.4, llvmorg-18.1.3, llvmorg-18.1.2, llvmorg-18.1.1, llvmorg-18.1.0, llvmorg-18.1.0-rc4, llvmorg-18.1.0-rc3, llvmorg-18.1.0-rc2, llvmorg-18.1.0-rc1, llvmorg-19-init, llvmorg-17.0.6, llvmorg-17.0.5, llvmorg-17.0.4, llvmorg-17.0.3, llvmorg-17.0.2, llvmorg-17.0.1, llvmorg-17.0.0, llvmorg-17.0.0-rc4, llvmorg-17.0.0-rc3, llvmorg-17.0.0-rc2, llvmorg-17.0.0-rc1, llvmorg-18-init, llvmorg-16.0.6, llvmorg-16.0.5, llvmorg-16.0.4, llvmorg-16.0.3, llvmorg-16.0.2, llvmorg-16.0.1, llvmorg-16.0.0, llvmorg-16.0.0-rc4, llvmorg-16.0.0-rc3, llvmorg-16.0.0-rc2, llvmorg-16.0.0-rc1, llvmorg-17-init, llvmorg-15.0.7, llvmorg-15.0.6, llvmorg-15.0.5, llvmorg-15.0.4, llvmorg-15.0.3, working, llvmorg-15.0.2, llvmorg-15.0.1, llvmorg-15.0.0, llvmorg-15.0.0-rc3, llvmorg-15.0.0-rc2, llvmorg-15.0.0-rc1, llvmorg-16-init, llvmorg-14.0.6, llvmorg-14.0.5, llvmorg-14.0.4, llvmorg-14.0.3, llvmorg-14.0.2, llvmorg-14.0.1, llvmorg-14.0.0, llvmorg-14.0.0-rc4, llvmorg-14.0.0-rc3, llvmorg-14.0.0-rc2, llvmorg-14.0.0-rc1, llvmorg-15-init, llvmorg-13.0.1, llvmorg-13.0.1-rc3, llvmorg-13.0.1-rc2, llvmorg-13.0.1-rc1, llvmorg-13.0.0, llvmorg-13.0.0-rc4, llvmorg-13.0.0-rc3, llvmorg-13.0.0-rc2, llvmorg-13.0.0-rc1, llvmorg-14-init, llvmorg-12.0.1, llvmorg-12.0.1-rc4, llvmorg-12.0.1-rc3, llvmorg-12.0.1-rc2, llvmorg-12.0.1-rc1, llvmorg-12.0.0, llvmorg-12.0.0-rc5, llvmorg-12.0.0-rc4, llvmorg-12.0.0-rc3, llvmorg-12.0.0-rc2, llvmorg-11.1.0, llvmorg-11.1.0-rc3, llvmorg-12.0.0-rc1, llvmorg-13-init, llvmorg-11.1.0-rc2, llvmorg-11.1.0-rc1, llvmorg-11.0.1, llvmorg-11.0.1-rc2, llvmorg-11.0.1-rc1, llvmorg-11.0.0, llvmorg-11.0.0-rc6, llvmorg-11.0.0-rc5, llvmorg-11.0.0-rc4, llvmorg-11.0.0-rc3, llvmorg-11.0.0-rc2, llvmorg-11.0.0-rc1, llvmorg-12-init, llvmorg-10.0.1, llvmorg-10.0.1-rc4, llvmorg-10.0.1-rc3, llvmorg-10.0.1-rc2, llvmorg-10.0.1-rc1, llvmorg-10.0.0, llvmorg-10.0.0-rc6, llvmorg-10.0.0-rc5, llvmorg-10.0.0-rc4, llvmorg-10.0.0-rc3, llvmorg-10.0.0-rc2, llvmorg-10.0.0-rc1, llvmorg-11-init, llvmorg-9.0.1, llvmorg-9.0.1-rc3, llvmorg-9.0.1-rc2, llvmorg-9.0.1-rc1, llvmorg-9.0.0, llvmorg-9.0.0-rc6, llvmorg-9.0.0-rc5, llvmorg-9.0.0-rc4, llvmorg-9.0.0-rc3, llvmorg-9.0.0-rc2, llvmorg-9.0.0-rc1, llvmorg-10-init, llvmorg-8.0.1, llvmorg-8.0.1-rc4, llvmorg-8.0.1-rc3, llvmorg-8.0.1-rc2, llvmorg-8.0.1-rc1, llvmorg-8.0.0, llvmorg-8.0.0-rc5, llvmorg-8.0.0-rc4, llvmorg-8.0.0-rc3, llvmorg-7.1.0, llvmorg-7.1.0-rc1, llvmorg-8.0.0-rc2, llvmorg-8.0.0-rc1, llvmorg-7.0.1, llvmorg-7.0.1-rc3, llvmorg-7.0.1-rc2, llvmorg-7.0.1-rc1, llvmorg-7.0.0, llvmorg-7.0.0-rc3, llvmorg-7.0.0-rc2, llvmorg-7.0.0-rc1, llvmorg-6.0.1, llvmorg-6.0.1-rc3, llvmorg-6.0.1-rc2, llvmorg-6.0.1-rc1, llvmorg-5.0.2, llvmorg-5.0.2-rc2, llvmorg-5.0.2-rc1, llvmorg-6.0.0, llvmorg-6.0.0-rc3, llvmorg-6.0.0-rc2, llvmorg-6.0.0-rc1, llvmorg-5.0.1, llvmorg-5.0.1-rc3, llvmorg-5.0.1-rc2, llvmorg-5.0.1-rc1, llvmorg-5.0.0, llvmorg-5.0.0-rc5, llvmorg-5.0.0-rc4, llvmorg-5.0.0-rc3, llvmorg-5.0.0-rc2, llvmorg-5.0.0-rc1, llvmorg-4.0.1, llvmorg-4.0.1-rc3, llvmorg-4.0.1-rc2, llvmorg-4.0.1-rc1, llvmorg-4.0.0, llvmorg-4.0.0-rc4, llvmorg-4.0.0-rc3, llvmorg-4.0.0-rc2, llvmorg-4.0.0-rc1, llvmorg-3.9.1, llvmorg-3.9.1-rc3, llvmorg-3.9.1-rc2, llvmorg-3.9.1-rc1, llvmorg-3.9.0, llvmorg-3.9.0-rc3, llvmorg-3.9.0-rc2, llvmorg-3.9.0-rc1, llvmorg-3.8.1, llvmorg-3.8.1-rc1, llvmorg-3.8.0, llvmorg-3.8.0-rc3, llvmorg-3.8.0-rc2, llvmorg-3.8.0-rc1, llvmorg-3.7.1, llvmorg-3.7.1-rc2, llvmorg-3.7.1-rc1, llvmorg-3.7.0, llvmorg-3.7.0-rc4, llvmorg-3.7.0-rc3, studio-1.4 |
|
| #
819f9ffe |
| 05-Aug-2015 |
Argyrios Kyrtzidis <akyrtzi@gmail.com> |
[test/Index] Update libclang tests to use libclang for creating PCH files.
This is consistent and tests the primary configuration we want to test, libclang
creating and consuming PCH files.
llvm-sv
[test/Index] Update libclang tests to use libclang for creating PCH files.
This is consistent and tests the primary configuration we want to test, libclang
creating and consuming PCH files.
llvm-svn: 244066
show more ...
|
|
Revision tags: llvmorg-3.7.0-rc2, llvmorg-3.7.0-rc1, llvmorg-3.6.2, llvmorg-3.6.2-rc1, llvmorg-3.6.1, llvmorg-3.6.1-rc1, llvmorg-3.5.2, llvmorg-3.5.2-rc1, llvmorg-3.6.0, llvmorg-3.6.0-rc4, llvmorg-3.6.0-rc3, llvmorg-3.6.0-rc2, llvmorg-3.6.0-rc1, llvmorg-3.5.1, llvmorg-3.5.1-rc2, llvmorg-3.5.1-rc1, llvmorg-3.5.0, llvmorg-3.5.0-rc4, llvmorg-3.5.0-rc3, llvmorg-3.5.0-rc2, llvmorg-3.5.0-rc1, llvmorg-3.4.2, llvmorg-3.4.2-rc1, llvmorg-3.4.1, llvmorg-3.4.1-rc2, llvmorg-3.4.1-rc1, llvmorg-3.4.0, llvmorg-3.4.0-rc3, llvmorg-3.4.0-rc2, llvmorg-3.4.0-rc1 |
|
| #
b534d3a0 |
| 26-Jul-2013 |
Argyrios Kyrtzidis <akyrtzi@gmail.com> |
[libclang] Remove comma from the blacklist of characters that prevent a comment to be attached to a decl.
It's common to use an availability function macro at the start of a decl.
rdar://13965065
l
[libclang] Remove comma from the blacklist of characters that prevent a comment to be attached to a decl.
It's common to use an availability function macro at the start of a decl.
rdar://13965065
llvm-svn: 187230
show more ...
|
|
Revision tags: llvmorg-3.3.1-rc1, llvmorg-3.3.0, llvmorg-3.3.0-rc3, llvmorg-3.3.0-rc2, llvmorg-3.3.0-rc1, llvmorg-3.2.0, llvmorg-3.2.0-rc3, llvmorg-3.2.0-rc2 |
|
| #
d947a66c |
| 15-Nov-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Split annotate-comments.cpp into a fragile (that uses hardcoded line numbers)
and a non-fragile (that uses [[@LINE]]) parts.
llvm-svn: 168098
|
|
Revision tags: llvmorg-3.2.0-rc1 |
|
| #
4edb74cf |
| 19-Oct-2012 |
NAKAMURA Takumi <geek4civic@gmail.com> |
clang/test/Index/annotate-comments.cpp: Relax the expression to be matched to -fms-compatibility. Then XFAIL can be removed.
FYI, it can be reproduced with "c-index-test -std=c++11 -fms-compatibilit
clang/test/Index/annotate-comments.cpp: Relax the expression to be matched to -fms-compatibility. Then XFAIL can be removed.
FYI, it can be reproduced with "c-index-test -std=c++11 -fms-compatibility".
llvm-svn: 166261
show more ...
|
| #
a9eebc31 |
| 19-Oct-2012 |
NAKAMURA Takumi <geek4civic@gmail.com> |
clang/test/Index/annotate-comments.cpp: Mark this as XFAIL on msvc. Investigating.
llvm-svn: 166250
|
| #
673c5215 |
| 18-Oct-2012 |
Fariborz Jahanian <fjahanian@apple.com> |
Fix this test to match recent addition of declaration tag.
llvm-svn: 166190
|
| #
a7d76d26 |
| 17-Oct-2012 |
Fariborz Jahanian <fjahanian@apple.com> |
[Doc parsing]: This patch adds <Declaration> tag to
XML comment for declarations which pretty-prints
declaration. I had to XFAIL one test annotate-comments.cpp.
This test is currently unmaintainable
[Doc parsing]: This patch adds <Declaration> tag to
XML comment for declarations which pretty-prints
declaration. I had to XFAIL one test annotate-comments.cpp.
This test is currently unmaintainable as written.
Dmitri G., can you see what we can do about this test.
We should change this test such that adding a new tag does not wreck
havoc to the test.
llvm-svn: 166130
show more ...
|
| #
7acbf00f |
| 10-Sep-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp.
Now we have a list of all commands. This is a good thing in itself, but it
also enables us to easily implement typo correcti
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp.
Now we have a list of all commands. This is a good thing in itself, but it
also enables us to easily implement typo correction for command names.
With this change we have objects that contain information about each command,
so it makes sense to resolve command name just once during lexing (currently we
store command names as strings and do a linear search every time some property
value is needed). Thus comment token and AST nodes were changed to contain a
command ID -- index into a tables of builtin and registered commands. Unknown
commands are registered during parsing and thus are also uniformly assigned an
ID. Using an ID instead of a StringRef is also a nice memory optimization
since ID is a small integer that fits into a common bitfield in Comment class.
This change implies that to get any information about a command (even a command
name) we need a CommandTraits object to resolve the command ID to CommandInfo*.
Currently a fresh temporary CommandTraits object is created whenever it is
needed since it does not have any state. But with this change it has state --
new commands can be registered, so a CommandTraits object was added to
ASTContext.
Also, in libclang CXComment has to be expanded to include a CXTranslationUnit
so that all functions working on comment AST nodes can get a CommandTraits
object. This breaks binary compatibility of CXComment APIs.
Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't
need TU parameter anymore, so it was removed. This is a source-incompatible
change for this C API.
llvm-svn: 163540
show more ...
|
| #
557a8d56 |
| 28-Aug-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Merging consecutive comments: be more conservative.
Should fix part 2 of PR13374.
llvm-svn: 162723
|
| #
75eea899 |
| 21-Aug-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
CommentBriefParser: allow paragraphs to be separated by line of whitespace.
Skip paragraphs that contain only whitespace.
llvm-svn: 162315
|
| #
bacbc08b |
| 09-Aug-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Remove absolute file path in test.
llvm-svn: 161602
|
| #
ba7aca3b |
| 09-Aug-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Comment to HTML and XML conversion: ignore commands that contain a declaration
as their argument. For example, \fn, \function, \typedef, \method, \class etc.
llvm-svn: 161601
|
| #
dcbc8ce2 |
| 09-Aug-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Comment to HTML and XML conversion: use CommandTraits to classify commands.
This also fixes a bug in comment to XML conversion: \result was just an
ordinary paragraph, not an alias for \returns.
ll
Comment to HTML and XML conversion: use CommandTraits to classify commands.
This also fixes a bug in comment to XML conversion: \result was just an
ordinary paragraph, not an alias for \returns.
llvm-svn: 161596
show more ...
|
| #
6cffc192 |
| 08-Aug-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Comment XML: use xml:space="preserve" in Verbatim tags, so that XML tidy does
not compress spaces in verbatim content.
llvm-svn: 161531
|
| #
168d2341 |
| 07-Aug-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Comment AST: DeclInfo: add a special kind for enums.
Comment XML: add a root node kind for enums.
llvm-svn: 161442
|
| #
94ef6357 |
| 07-Aug-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Comment AST: treat enumerators as "variables" in DeclInfo.
llvm-svn: 161435
|
| #
740c0fbe |
| 07-Aug-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
libclang API for comment-to-xml conversion.
The implementation also includes a Relax NG schema and tests for the schema
itself. The schema is used in c-index-test to verify that XML documents we
pr
libclang API for comment-to-xml conversion.
The implementation also includes a Relax NG schema and tests for the schema
itself. The schema is used in c-index-test to verify that XML documents we
produce are valid. In order to do the validation, we add an optional libxml2
dependency for c-index-test.
Credits for CMake part go to Doug Gregor. Credits for Autoconf part go to Eric
Christopher. Thanks!
llvm-svn: 161431
show more ...
|
| #
58e41319 |
| 01-Aug-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Comment to HTML conversion: correct typo in CSS class name: taram -> tparam
llvm-svn: 161145
|
| #
307cf89b |
| 01-Aug-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Comment to HTML conversion: skip \tparam commands with whitespace paragraphs
llvm-svn: 161096
|
| #
7c0456f9 |
| 01-Aug-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Comment to HTML conversion: escape HTML special characters in command arguments
llvm-svn: 161094
|
| #
34df2204 |
| 31-Jul-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Comment parsing: add support for \tparam command on all levels.
The only caveat is renumbering CXCommentKind enum for aesthetic reasons -- this
breaks libclang binary compatibility, but should not b
Comment parsing: add support for \tparam command on all levels.
The only caveat is renumbering CXCommentKind enum for aesthetic reasons -- this
breaks libclang binary compatibility, but should not be a problem since API is
so new.
This also fixes PR13372 as a side-effect.
llvm-svn: 161087
show more ...
|
| #
4586df76 |
| 27-Jul-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
Implement resolving of HTML character references (named: &, decimal: *,
hex: ) during comment parsing.
Now internal representation of plain text in comment AST does not contain
charact
Implement resolving of HTML character references (named: &, decimal: *,
hex: ) during comment parsing.
Now internal representation of plain text in comment AST does not contain
character references, but the characters themselves.
llvm-svn: 160891
show more ...
|
| #
6b375193 |
| 25-Jul-2012 |
Dmitri Gribenko <gribozavr@gmail.com> |
libclang comment to HTML rendering: \result is the same as \returns
llvm-svn: 160738
|