8

I am trying to use unicode variable names in g++.

It does not appear to work.

Does g++ not support unicode variable names, ... or is there some subset of unicode (from which I'm not testing in).

Thanks!

anon
  • 38,809
  • 49
  • 191
  • 291
  • ¤ g++ is just not standard-conforming wrt. characters in identifiers. But I don't know of any compiler that *is* conforming. It is my impression that most compilers limit the identifier characters to English A...Z and underscore, plus $ sign, which is wrong in two ways: not allowing the huge range of Unicode characters specified in Annex E of the standard (I've listed them at http://pastie.org/3110152), and allowing $, which the standard does not allow. In short, the standard and existing practice is very much at odds. Perhaps with C++11... ;-) Cheers & hth., – Cheers and hth. - Alf Jan 02 '12 at 03:33
  • 1
    @Cheersandhth.-Alf Try clang :) – Richard Smith Jul 24 '13 at 07:24

2 Answers2

10

You have to specify the -fextended-identifiers flag when compiling, you also have to use \uXXXX or \uXXXXXXXX for unicode(atleast in gcc it's unicode)

Identifiers (variable/class names etc) in g++ can't be of utf-8/utf-16 or whatever encoding, they have to be:

identifier:
  nondigit
  identifier nondigit
  identifier digit

a nondigit is

nondigit: one of
  universalcharactername
  _ a b c d e f g h i j k l m n o p q r s t u v w x y z
  A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

and a universalcharactername is

universalcharactername:
  \UXXXXXXXX
  \uXXXX

Thus, if you save your source file as UTF-8, you cannot have a variable like e.g.:

int høyde = 10;

it had to be written like:

int h\u00F8yde = 10;

(which imo would beat the whole purpose - so just stick with a-z)

Zulan
  • 21,516
  • 6
  • 45
  • 98
nos
  • 215,098
  • 54
  • 400
  • 488
  • 3
    Is there better support in clang? – anon Apr 21 '10 at 10:27
  • I don't know, but you should ask another question for that. – nos Apr 21 '10 at 11:25
  • 1
    g++ is not standard-conforming here (but neither are other compilers, including Comeau). For standard C++, in the very first phase of translation "Any source file character not in the basic source character set (2.3) is replaced by the universal-character-name that designates that character", and the lexer rules operate on the result of that. In the C++11 standard this is specified in "Phases of translation" §2.2/1 1st list item. – Cheers and hth. - Alf Jan 02 '12 at 03:23
  • 2
    @anon Yes, clang allows accented characters in identifiers. – Richard Smith Jul 24 '13 at 07:23
  • 3
    @anon **yes**, from clang 3.3 onwards [there is](http://llvm.org/releases/3.3/tools/clang/docs/ReleaseNotes.html#extended-identifiers-unicode-support-and-universal-character-names) support for unicode identifiers right in UTF-8. – ulidtko Mar 20 '14 at 17:19
  • 1
    9 years later G++ 9.1 is _still_ blind to UTF-8 symbols, even with `-fextended-identifiers -finput-charset=UTF-8`. (For reference, also MSVC++ does fine, either with -utf-8 or with a BOM in the source.) See also: https://stackoverflow.com/a/12693346/1479945 – Sz. Aug 13 '19 at 10:59
4

A one-line patch to the cpp preprocessor allows UTF-8 input. Details for gcc are given at

https://www.raspberrypi.org/forums/viewtopic.php?p=802657

however, since the preprocessor is shared, the same patch should work for g++ as well. In particular, the patch needed, as of gcc-5.2 is

diff -cNr gcc-5.2.0/libcpp/charset.c gcc-5.2.0-ejo/libcpp/charset.c
*** gcc-5.2.0/libcpp/charset.c  Mon Jan  5 04:33:28 2015
--- gcc-5.2.0-ejo/libcpp/charset.c  Wed Aug 12 14:34:23 2015
***************
*** 1711,1717 ****
    struct _cpp_strbuf to;
    unsigned char *buffer;

!   input_cset = init_iconv_desc (pfile, SOURCE_CHARSET, input_charset);
    if (input_cset.func == convert_no_conversion)
      {
        to.text = input;
--- 1711,1717 ----
    struct _cpp_strbuf to;
    unsigned char *buffer;

!   input_cset = init_iconv_desc (pfile, "C99", input_charset);
    if (input_cset.func == convert_no_conversion)
      {
        to.text = input;

Note that for the above patch to work, a recent version of iconv needs to be installed that supports C99 conversions. Type iconv --list to verify this, otherwise, you can install a new version of iconv along with gcc as described in the link above. Change the configure command to

$ ../gcc-5.2.0/configure -v --disable-multilib \
    --with-libiconv-prefix=/usr/local/gcc-5.2 \
    --prefix=/usr/local/gcc-5.2 \
    --enable-languages="c,c++"

if you are building for x86 and want to include the c++ compiler as well.

ejolson
  • 146
  • 4