How to check the charset of string?

Question

How do I check if the charset of a string is UTF8?

score 33 · Answer 1 · answered Jun 24 '09 at 09:44

33

Don't reinvent the wheel. There is a builtin function for that task: mb_check_encoding().

mb_check_encoding($string, 'UTF-8');

answered Jun 24 '09 at 09:44

soulmerge

73,842
19
118
155

What happens if you don't have that extension installed? – CMCDragonkai Nov 07 '14 at 14:42
2

@CMCDragonkai If your data is in a multibyte format (like UTF-8), you should definitely install the PHP extension that provides the functions to correctly operate on your input. If you use `strlen` on an UTF-8 string, for example, you will get the wrong length if the input contains non-ASCII characters. – soulmerge Nov 12 '14 at 09:20

score 15 · Answer 2 · edited May 23 '17 at 12:17

Just a side note:

You cannot determine if a given string is encoded in UTF-8. You only can determine if a given string is definitively not encoded in UTF-8. Please see a related question here:

You cannot detect if a given string (or byte sequence) is a UTF-8 encoded text as for example each and every series of UTF-8 octets is also a valid (if nonsensical) series of Latin-1 (or some other encoding) octets. However not every series of valid Latin-1 octets are valid UTF-8 series.

score 7 · Accepted Answer · answered Jun 24 '09 at 09:40

function is_utf8($string) {   
return preg_match('%^(?:  
[\x09\x0A\x0D\x20-\x7E] # ASCII  
| [\xC2-\xDF][\x80-\xBF] # non-overlong 2-byte  
| \xE0[\xA0-\xBF][\x80-\xBF] # excluding overlongs  
| [\xE1-\xEC\xEE\xEF][\x80-\xBF]{2} # straight 3-byte  
| \xED[\x80-\x9F][\x80-\xBF] # excluding surrogates  
| \xF0[\x90-\xBF][\x80-\xBF]{2} # planes 1-3  
| [\xF1-\xF3][\x80-\xBF]{3} # planes 4-15  
| \xF4[\x80-\x8F][\x80-\xBF]{2} # plane 16  
)*$%xs', $string);

}

I have checked. This function is effective.

score 7 · Answer 4 · answered Jun 24 '09 at 10:27

7

Better yet, use both of the above solutions.

function isUtf8($string) {
    if (function_exists("mb_check_encoding") && is_callable("mb_check_encoding")) {
        return mb_check_encoding($string, 'UTF8');
    }

    return preg_match('%^(?:
          [\x09\x0A\x0D\x20-\x7E]            # ASCII
        | [\xC2-\xDF][\x80-\xBF]             # non-overlong 2-byte
        |  \xE0[\xA0-\xBF][\x80-\xBF]        # excluding overlongs
        | [\xE1-\xEC\xEE\xEF][\x80-\xBF]{2}  # straight 3-byte
        |  \xED[\x80-\x9F][\x80-\xBF]        # excluding surrogates
        |  \xF0[\x90-\xBF][\x80-\xBF]{2}     # planes 1-3
        | [\xF1-\xF3][\x80-\xBF]{3}          # planes 4-15
        |  \xF4[\x80-\x8F][\x80-\xBF]{2}     # plane 16
    )*$%xs', $string);

}

answered Jun 24 '09 at 10:27

nikc.org

16,462
6
50
83

2

Call to both function_exists() and is_callable() is redundant, you can skip one of them. But since is_callable() is intended to check if the parameter is a valid callback (see link below), I would use function_exists() to make it more readable. Link to callback pseudo-type: http://php.net/manual/en/language.pseudo-types.php#language.types.callback – soulmerge Jul 28 '09 at 07:21
@nikc.org - I don't understand - why do you want to test whether the function `mb_check_encoding` exists? – JDelage Feb 19 '12 at 19:39
2

@JDelage because it's part of an extension (php.net/manual/en/book.mbstring.php) that is not guaranteed to always be present. – nikc.org Feb 20 '12 at 07:12

Jyothish V · Answer 5 · 2017-08-02T11:47:33.910

1

mb_detect_encoding($string); will return the actual character set of $string. mb_check_encoding($string, 'UTF-8'); will return TRUE if character set of $string is UTF-8 else FALSE

edited Aug 02 '17 at 11:47

answered Sep 25 '14 at 11:41

Jyothish V

66
9

score 0 · Answer 6 · answered Jun 24 '09 at 09:42

0

if its send to u from server

echo $_SERVER['HTTP_ACCEPT_CHARSET'];

answered Jun 24 '09 at 09:42

Haim Evgi

123,187
45
217
223

1

You can have UTF-8 **and** ISO encoded strings within 1 request/file. How would HTTP headers help in any case? – Daniel W. Jan 22 '14 at 10:53

score 0 · Answer 7 · answered Aug 09 '13 at 14:23

None of the above answers are correct. Yes, they may be working. If you take the answer with the preg_replace function, are you trying to kill your server if you process a lot of stirng ? Use this pure PHP function with no regex, work 100% of the time and it's way faster.

if(function_exists('grk_Is_UTF8') === FALSE){
    function grk_Is_UTF8($String=''){
        #   On va calculer la longeur de la chaîne
        $Len = strlen($String);

        #   On va boucler sur chaque caractère
        for($i = 0; $i < $Len; $i++){
            #   On va aller chercher la valeur ASCII du caractère
            $Ord = ord($String[$i]);
            if($Ord > 128){
                if($Ord > 247){
                    return FALSE;
                } elseif($Ord > 239){
                    $Bytes = 4;
                } elseif($Ord > 223){
                    $Bytes = 3;
                } elseif($Ord > 191){
                    $Bytes = 2;
                } else {
                    return FALSE;
                }

                #   
                if(($i + $Bytes) > $Len){
                    return FALSE;
                }

                #   On va boucler sur chaque bytes / caractères
                while($Bytes > 1){
                    #   +1
                    $i++;

                    #   On va aller chercher la valeur ASCII du caractère / byte
                    $Ord = ord($String[$i]);
                    if($Ord < 128 OR $Ord > 191){
                        return FALSE;
                    }

                    #   Parfait
                    $Bytes--;
                }
            }
        }

        #   Vrai
        return TRUE;
    }
}

How to check the charset of string?

7 Answers7

Linked