\w in PHP preg_replace covers only second byte of UTF-8 chars

Question

we have this code:

$value = preg_replace("/[^\w]/", '', $value);

where $value is in utf-8. After this transformation first byte of multibyte characters is stripped. How to make \w cover UTF-8 chars completely?

Sorry, i am not very well in PHP

Answer 1

You could try with the /u modifier:

This modifier turns on additional functionality of PCRE that is incompatible with Perl. Pattern strings are treated as UTF-8. This modifier is available from PHP 4.1.0 or greater on Unix and from PHP 4.2.3 on win32. UTF-8 validity of the pattern is checked since PHP 4.3.5.

If that won't do, try

• mb_ereg_replace - Replace regular expression with multibyte support

instead.

Answer 2

There is this nasty u modifier to pcre patterns in PHP. It states that the regex is encoded in UTF8, but I found that it treats the input as UTF8, too.

Answer 3

Append u to regex, to turn on the multibyte unicode mode of PCRE:

$value = preg_replace("/[^\w]/u", '', $value);

Corollary

In unicode mode, PCRE expects everything is multibyte and if it is not then there will be problems meeting deadlines. Therefore, to convert anything to UTF-8 (and drop any unconvertible junk), we first use:

$value = iconv( 'ISO-8859-1', 'UTF-8//IGNORE//TRANSLIT', $i );

to clean and prep the input.

Because everything can be encoded into ISO-8859-1 (even if some obscure characters appear incorrectly), and since most web browsers run natively in 8859 (unless told to use UTF-8), we've found this function as a general, safe, effective method to 'take anything, drop any junk, and convert into UTF-8'.

mb_ereg_* is deprecated as of 5.3.0 -- so using those functions is not the right way to go.

Answer 4

try this function instead...http://php.net/manual/en/function.mb-ereg-replace.php

Answer 5

Use [^\w]+ instead of [^\w]

You can also use \W in place of [^\w]