[Windows] Convert from UTF-8 to UTF-16 when writing to a Windows console

Summary:
Calling WriteConsoleW is the most reliable way to print Unicode
characters to a Windows console.

If binary data get

[Windows] Convert from UTF-8 to UTF-16 when writing to a Windows console

Summary:
Calling WriteConsoleW is the most reliable way to print Unicode
characters to a Windows console.

If binary data gets printed to the console, attempting to re-encode it
shouldn't be a problem, since garbage in can produce garbage out.

This breaks printing strings in the local codepage, which WriteConsoleA
knows how to handle. For example, this can happen when user source code
is encoded with the local codepage, and an LLVM tool quotes it while
emitting a caret diagnostic. This is unfortunate, but well-behaved tools
should validate that their input is UTF-8 and escape non-UTF-8
characters before sending them to raw_fd_ostream. Clang already does
this, but not all LLVM tools do this.

One drawback to the current implementation is printing a string a byte
at a time doesn't work. Consider this LLVM code:
for (char C : MyStr) outs() << C;

Because outs() is now unbuffered, we wil try to convert each byte to
UTF-16, which will fail. However, this already didn't work, so I think
we may as well update callers that do that as we find them to print
complete portions of strings. You can see a real example of this in my
patch to SourceMgr.cpp

Fixes PR38669 and PR36267.

Reviewers: zturner, efriedma

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D51558

llvm-svn: 341433

show more ...

[Support] Fix some Wundef warnings

For the _WIN32 macro, it is the definedness that matters rather than
the value. Most uses of the macro already rely on the definedness.
This commit fixes the few

[Support] Fix some Wundef warnings

For the _WIN32 macro, it is the definedness that matters rather than
the value. Most uses of the macro already rely on the definedness.
This commit fixes the few remaining uses that relied on the value.

Differential Revision: https://reviews.llvm.org/D51105

llvm-svn: 340520

show more ...

IWYU for llvm-config.h, removals. Also see r331184.

llvm-svn: 331190

s/LLVM_ON_WIN32/_WIN32/, llvm

LLVM_ON_WIN32 is set exactly with MSVC and MinGW (but not Cygwin) in
HandleLLVMOptions.cmake, which is where _WIN32 defined too. Just use the
default macro instead of

s/LLVM_ON_WIN32/_WIN32/, llvm

LLVM_ON_WIN32 is set exactly with MSVC and MinGW (but not Cygwin) in
HandleLLVMOptions.cmake, which is where _WIN32 defined too. Just use the
default macro instead of a reinvented one.

See thread "Replacing LLVM_ON_WIN32 with just _WIN32" on llvm-dev and cfe-dev.
No intended behavior change.

This moves over all uses of the macro, but doesn't remove the definition
of it in (llvm-)config.h yet.

llvm-svn: 331127

show more ...

[NFC] Header cleanup

Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedM

[NFC] Header cleanup

Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'

Patch by Eugene Kosov <claprix@yandex.ru>

Differential Revision: http://reviews.llvm.org/D19219

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266595

show more ...

Add #include llvm-config.h to Locale.cpp which depends on LLVM_ON_WIN32.
Source code was assuming that llvm-config.h would be included somehow but
up to r247253 that added #include "llvm/Support/Comp

Add #include llvm-config.h to Locale.cpp which depends on LLVM_ON_WIN32.
Source code was assuming that llvm-config.h would be included somehow but
up to r247253 that added #include "llvm/Support/Compiler.h" to StringRef.h
the config file was not actually included. The inclusion of llvm-config.h
caused a change of behaviour in tools/clang/test/Frontend/source-col-map.c:
previously it would output the original UTF-8 but now it outputs <U+03B1>.

llvm-svn: 247409

show more ...

Fix "the the" in comments.

llvm-svn: 240112

Move generic isPrint and columnWidth implementations to a separate header/source to allow using both generic and system-dependent versions on win32.

Summary:
This is needed so we can use generic col

Move generic isPrint and columnWidth implementations to a separate header/source to allow using both generic and system-dependent versions on win32.

Summary:
This is needed so we can use generic columnWidthUTF8 in clang-format on
win32 simultaneously with a separate system-dependent implementations of
isPrint/columnWidth in TextDiagnostic.cpp to avoid attempts to print Unicode
characters using narrow-character interfaces (which is not supported on Windows,
and we'll have to figure out how to handle this).

Reviewers: jordan_rose

Reviewed By: jordan_rose

CC: llvm-commits, klimek

Differential Revision: http://llvm-reviews.chandlerc.com/D1559

llvm-svn: 189952

show more ...

platform support for counting column widths and checking isprint

llvm-svn: 154944