[Pseudo] Token/TokenStream, PP directive parser.

The TokenStream class is the representation of the source code that will
be fed into the GLR parser.

This patch allows a "raw" TokenStream to be bui

[Pseudo] Token/TokenStream, PP directive parser.

The TokenStream class is the representation of the source code that will
be fed into the GLR parser.

This patch allows a "raw" TokenStream to be built by reading source code.
It also supports scanning a TokenStream to find the directive structure.

Next steps (with placeholders in the code): heuristically choosing a
path through #ifs, preprocessing the code by stripping directives and comments.
These will produce a suitable stream to feed into the parser proper.

Differential Revision: https://reviews.llvm.org/D119162

show more ...

Use switch instead of series of comparisons

This is style correction, no functional changes.

Differential Revision: https://reviews.llvm.org/D65670

llvm-svn: 367759

[Parser] Use special definition for pragma annotations

Previously pragma annotation tokens were described as any other
annotations in TokenKinds.def. This change introduces special macro
PRAGMA_ANNO

[Parser] Use special definition for pragma annotations

Previously pragma annotation tokens were described as any other
annotations in TokenKinds.def. This change introduces special macro
PRAGMA_ANNOTATION for the pragma descriptions. It allows implementing
checks that deal with pragma annotations only.

Differential Revision: https://reviews.llvm.org/D65405

llvm-svn: 367575

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[C++11] Use 'nullptr'.

llvm-svn: 208280

Prospective MSVC 2010 build fix

Try to fix Compiler Error C2011 following r198607 by removing enum from 'enum
TokenKind' parameter types.

llvm-svn: 198621

Support diagnostic formatting of keyword tokens

Implemented with a new getKeywordSpelling() accessor. Unlike getTokenName() the
result of this function is stable and may be used in diagnostic output

Support diagnostic formatting of keyword tokens

Implemented with a new getKeywordSpelling() accessor. Unlike getTokenName() the
result of this function is stable and may be used in diagnostic output.

Uses of this feature are split out into the subsequent commit.

llvm-svn: 198604

show more ...

Rename getTokenSimpleSpelling() to getPunctuatorSpelling()

That's what it does, what the documentation says it does and what callers
expect it to do.

llvm-svn: 198603

Apply some LLVM_READONLY / LLVM_READNONE on diagnostic functions

llvm-svn: 198598

Sort all of Clang's files under 'lib', and fix up the broken headers
uncovered.

This required manually correcting all of the incorrect main-module
headers I could find, and running the new llvm/util

Sort all of Clang's files under 'lib', and fix up the broken headers
uncovered.

This required manually correcting all of the incorrect main-module
headers I could find, and running the new llvm/utils/sort_includes.py
script over the files.

I also manually added quite a few missing headers that were uncovered by
shuffling the order or moving headers up to be main-module-headers.

llvm-svn: 169237

show more ...

This patch adds a PUNCTUATOR macro (specialization of TOK) in TokenKinds.def and makes use of it in tok::getTokenSimpleSpelling.

llvm-svn: 90042

Clean up and document code modification hints.

llvm-svn: 65641

Introduce code modification hints into the diagnostics system. When we
know how to recover from an error, we can attach a hint to the
diagnostic that states how to modify the code, which can be one o

Introduce code modification hints into the diagnostics system. When we
know how to recover from an error, we can attach a hint to the
diagnostic that states how to modify the code, which can be one of:

- Insert some new code (a text string) at a particular source
location
- Remove the code within a given range
- Replace the code within a given range with some new code (a text
string)

Right now, we use these hints to annotate diagnostic information. For
example, if one uses the '>>' in a template argument in C++98, as in
this code:

template<int I> class B { };
B<1000 >> 2> *b1;

we'll warn that the behavior will change in C++0x. The fix is to
insert parenthese, so we use code insertion annotations to illustrate
where the parentheses go:

test.cpp:10:10: warning: use of right-shift operator ('>>') in template
argument will require parentheses in C++0x
B<1000 >> 2> *b1;
^
( )


Use of these annotations is partially implemented for HTML
diagnostics, but it's not (yet) producing valid HTML, which may be
related to PR2386, so it has been #if 0'd out.

In this future, we could consider hooking this mechanism up to the
rewriter to actually try to fix these problems during compilation (or,
after a compilation whose only errors have fixes). For now, however, I
suggest that we use these code modification hints whenever we can, so
that we get better diagnostics now and will have better coverage when
we find better ways to use this information.

This also fixes PR3410 by placing the complaint about missing tokens
just after the previous token (rather than at the location of the next
token).

llvm-svn: 65570

show more ...

Make a major restructuring of the clang tree: introduce a top-level
lib dir and move all the libraries into it. This follows the main
llvm tree, and allows the libraries to be built in parallel. Th

Make a major restructuring of the clang tree: introduce a top-level
lib dir and move all the libraries into it. This follows the main
llvm tree, and allows the libraries to be built in parallel. The
top level now enforces that all the libs are built before Driver,
but we don't care what order the libs are built in. This speeds
up parallel builds, particularly incremental ones.

llvm-svn: 48402

show more ...