‪TYPO3CMS  11.5
TYPO3\CMS\Core\Charset\CharsetConverter Class Reference
Inheritance diagram for TYPO3\CMS\Core\Charset\CharsetConverter:

Public Member Functions

string conv ($inputString, $fromCharset, $toCharset)
string utf8_encode ($str, $charset)
string utf8_decode ($str, $charset, $useEntityForNoChar=false)
array utf8_to_numberarray ($str)
string UnumberToChar ($unicodeInteger)
int utf8CharToUnumber ($str, $hex=false)
string specCharsToASCII ($charset, $string)
string sb_char_mapping ($str, $charset)
string utf8_char_mapping ($str)
string euc_char_mapping ($str, $charset)

Protected Member Functions

int initCharset ($charset)
int initUnicodeData ()
int initToASCII ($charset)

Protected Attributes

int $noCharByteVal = 63
array $parsedCharsets = array( )
array $toASCII = array( )
array $twoByteSets
array $eucBasedSets

Detailed Description

Notes on UTF-8

Functions working on UTF-8 strings:

  • ‪strchr/strstr
  • ‪strrchr
  • ‪substr_count
  • ‪implode/explode/join

Functions nearly working on UTF-8 strings:

  • ‪trim/ltrim/rtrim: the second parameter 'charlist' won't work for characters not contained in 7-bit ASCII
  • ‪htmlentities: charset support for UTF-8 only since PHP 4.3.0
  • ‪preg_*: Support compiled into PHP by default nowadays, but could be unavailable, need to use modifier

Functions NOT working on UTF-8 strings:

  • ‪str*cmp
  • ‪stristr
  • ‪stripos
  • ‪substr
  • ‪strrev
  • ‪split/spliti
  • ‪... Class for conversion between charsets

Definition at line 53 of file CharsetConverter.php.

Member Function Documentation

◆ conv()

string TYPO3\CMS\Core\Charset\CharsetConverter::conv (   $inputString,

Convert from one charset to another charset.

string$inputString‪Input string
string$fromCharset‪From charset (the current charset of the string)
string$toCharset‪To charset (the output charset wanted)
‪string Converted string

Definition at line 105 of file CharsetConverter.php.

References TYPO3\CMS\Core\Charset\CharsetConverter\utf8_decode(), and TYPO3\CMS\Core\Charset\CharsetConverter\utf8_encode().

◆ euc_char_mapping()

string TYPO3\CMS\Core\Charset\CharsetConverter::euc_char_mapping (   $str,

Maps all characters of a string in the EUC charset family.

string$str‪EUC multibyte character string
string$charset‪The charset
‪string The converted string

Definition at line 800 of file CharsetConverter.php.

References TYPO3\CMS\Core\Charset\CharsetConverter\initToASCII().

Referenced by TYPO3\CMS\Core\Charset\CharsetConverter\specCharsToASCII().

◆ initCharset()

int TYPO3\CMS\Core\Charset\CharsetConverter::initCharset (   $charset)

This will initialize a charset for use if it's defined in the 'typo3/sysext/core/Resources/Private/Charsets/csconvtbl/' folder This function is automatically called by the conversion functions

PLEASE SEE: http://www.unicode.org/Public/MAPPINGS/

string$charset‪The charset to be initialized. Use lowercase charset always (the charset must match exactly with a filename in csconvtbl/ folder ([charset].tbl)
‪int Returns '1' if already loaded, '2' if the charset conversion table was found and parsed.
UnknownCharsetException‪if no charset table was found

Definition at line 415 of file CharsetConverter.php.

References TYPO3\CMS\Core\Utility\ExtensionManagementUtility\extPath(), TYPO3\CMS\Core\Core\Environment\getVarPath(), TYPO3\CMS\Core\Utility\GeneralUtility\trimExplode(), TYPO3\CMS\Core\Charset\CharsetConverter\UnumberToChar(), and TYPO3\CMS\Core\Utility\GeneralUtility\writeFileToTypo3tempDir().

Referenced by TYPO3\CMS\Core\Charset\CharsetConverter\initToASCII(), TYPO3\CMS\Core\Charset\CharsetConverter\utf8_decode(), and TYPO3\CMS\Core\Charset\CharsetConverter\utf8_encode().

◆ initToASCII()

int TYPO3\CMS\Core\Charset\CharsetConverter::initToASCII (   $charset)

This function initializes the to-ASCII conversion table for a charset other than UTF-8. This function is automatically called by the ASCII transliteration functions.

string$charsetCharset for which to initialize conversion.
‪int Returns FALSE on error, a TRUE value on success: 1 table already loaded, 2, cached version, 3 table parsed (and cached).

Definition at line 646 of file CharsetConverter.php.

References TYPO3\CMS\Core\Core\Environment\getVarPath(), TYPO3\CMS\Core\Charset\CharsetConverter\initCharset(), TYPO3\CMS\Core\Charset\CharsetConverter\initUnicodeData(), TYPO3\CMS\Core\Charset\CharsetConverter\utf8_decode(), and TYPO3\CMS\Core\Utility\GeneralUtility\writeFileToTypo3tempDir().

Referenced by TYPO3\CMS\Core\Charset\CharsetConverter\euc_char_mapping(), and TYPO3\CMS\Core\Charset\CharsetConverter\sb_char_mapping().

◆ initUnicodeData()

int TYPO3\CMS\Core\Charset\CharsetConverter::initUnicodeData ( )

◆ sb_char_mapping()

string TYPO3\CMS\Core\Charset\CharsetConverter::sb_char_mapping (   $str,

Maps all characters of a string in a single byte charset.

string$str‪The string
string$charset‪The charset
‪string The converted string

Definition at line 717 of file CharsetConverter.php.

References TYPO3\CMS\Core\Charset\CharsetConverter\initToASCII().

Referenced by TYPO3\CMS\Core\Charset\CharsetConverter\specCharsToASCII().

◆ specCharsToASCII()

string TYPO3\CMS\Core\Charset\CharsetConverter::specCharsToASCII (   $charset,

Converts special chars (like æøåÆØÅ, umlauts etc) to ascii equivalents (usually double-bytes, like æ => ae etc.)

string$charset‪Character set of string
string$string‪Input string to convert
‪string The converted string

Definition at line 692 of file CharsetConverter.php.

References TYPO3\CMS\Core\Charset\CharsetConverter\euc_char_mapping(), TYPO3\CMS\Core\Charset\CharsetConverter\sb_char_mapping(), and TYPO3\CMS\Core\Charset\CharsetConverter\utf8_char_mapping().

◆ UnumberToChar()

string TYPO3\CMS\Core\Charset\CharsetConverter::UnumberToChar (   $unicodeInteger)

Converts a UNICODE number to a UTF-8 multibyte character Algorithm based on script found at From: http://czyborra.com/utf/ Unit-tested by Kasper

The binary representation of the character's integer value is thus simply spread across the bytes and the number of high bits set in the lead byte announces the number of bytes in the multibyte sequence:

bytes | bits | representation 1 | 7 | 0vvvvvvv 2 | 11 | 110vvvvv 10vvvvvv 3 | 16 | 1110vvvv 10vvvvvv 10vvvvvv 4 | 21 | 11110vvv 10vvvvvv 10vvvvvv 10vvvvvv 5 | 26 | 111110vv 10vvvvvv 10vvvvvv 10vvvvvv 10vvvvvv 6 | 31 | 1111110v 10vvvvvv 10vvvvvv 10vvvvvv 10vvvvvv 10vvvvvv

int$unicodeInteger‪UNICODE integer
‪string UTF-8 multibyte character string
See also

Definition at line 327 of file CharsetConverter.php.

Referenced by TYPO3\CMS\Core\Charset\CharsetConverter\initCharset(), and TYPO3\CMS\Core\Charset\CharsetConverter\initUnicodeData().

◆ utf8_char_mapping()

string TYPO3\CMS\Core\Charset\CharsetConverter::utf8_char_mapping (   $str)

Maps all characters of a UTF-8 string.

string$str‪UTF-8 string
‪string The converted string

Definition at line 748 of file CharsetConverter.php.

References TYPO3\CMS\Core\Charset\CharsetConverter\initUnicodeData().

Referenced by TYPO3\CMS\Core\Charset\CharsetConverter\specCharsToASCII().

◆ utf8_decode()

string TYPO3\CMS\Core\Charset\CharsetConverter::utf8_decode (   $str,
  $useEntityForNoChar = false 

Converts $str from UTF-8 to $charset

string$str‪String in UTF-8 to convert to local charset
string$charsetCharset, lowercase. Must be found in csconvtbl/ folder.
bool$useEntityForNoChar‪If set, then characters that are not available in the destination character set will be encoded as numeric entities
‪string Output string, converted to local charset

Definition at line 196 of file CharsetConverter.php.

References TYPO3\CMS\Core\Charset\CharsetConverter\initCharset(), and TYPO3\CMS\Core\Charset\CharsetConverter\utf8CharToUnumber().

Referenced by TYPO3\CMS\Core\Charset\CharsetConverter\conv(), and TYPO3\CMS\Core\Charset\CharsetConverter\initToASCII().

◆ utf8_encode()

string TYPO3\CMS\Core\Charset\CharsetConverter::utf8_encode (   $str,

Converts $str from $charset to UTF-8

string$str‪String in local charset to convert to UTF-8
string$charsetCharset, lowercase. Must be found in csconvtbl/ folder.
‪string Output string, converted to UTF-8

Definition at line 134 of file CharsetConverter.php.

References TYPO3\CMS\Core\Charset\CharsetConverter\initCharset().

Referenced by TYPO3\CMS\Core\Charset\CharsetConverter\conv().

◆ utf8_to_numberarray()

array TYPO3\CMS\Core\Charset\CharsetConverter::utf8_to_numberarray (   $str)

Converts all chars in the input UTF-8 string into integer numbers returned in an array. All HTML entities (like & or £ or { or 㽝) will be detected as characters. Also, instead of integer numbers the real UTF-8 char is returned.

string$str‪Input string, UTF-8
‪array Output array with the char numbers

Definition at line 265 of file CharsetConverter.php.

◆ utf8CharToUnumber()

int TYPO3\CMS\Core\Charset\CharsetConverter::utf8CharToUnumber (   $str,
  $hex = false 

Converts a UTF-8 Multibyte character to a UNICODE number Unit-tested by Kasper

string$str‪UTF-8 multibyte character string
bool$hex‪If set, then a hex. number is returned.
‪int UNICODE integer
See also

Definition at line 373 of file CharsetConverter.php.

Referenced by TYPO3\CMS\Core\Charset\CharsetConverter\utf8_decode().

Member Data Documentation

◆ $eucBasedSets

array TYPO3\CMS\Core\Charset\CharsetConverter::$eucBasedSets
Initial value:
= array(
'gb2312' => 1,
'big5' => 1,
'euc-kr' => 1,
'shift_jis' => 1,

This tells the converter which charsets use a scheme like the Extended Unix Code:

Definition at line 85 of file CharsetConverter.php.

◆ $noCharByteVal

int TYPO3\CMS\Core\Charset\CharsetConverter::$noCharByteVal = 63

ASCII Value for chars with no equivalent.

Definition at line 59 of file CharsetConverter.php.

◆ $parsedCharsets

array TYPO3\CMS\Core\Charset\CharsetConverter::$parsedCharsets = array( )

This is the array where parsed conversion tables are stored (cached)

Definition at line 65 of file CharsetConverter.php.

◆ $toASCII

array TYPO3\CMS\Core\Charset\CharsetConverter::$toASCII = array( )

An array where charset-to-ASCII mappings are stored (cached)

Definition at line 71 of file CharsetConverter.php.

◆ $twoByteSets

array TYPO3\CMS\Core\Charset\CharsetConverter::$twoByteSets
Initial value:
= array(
'ucs-2' => 1,

This tells the converter which charsets has two bytes per char:

Definition at line 77 of file CharsetConverter.php.