17.20 - UTF8 Multibyte Sequences - Analytics Database - Teradata Vantage

Teradata Vantageā„¢ - Analytics Database International Character Set Support - 17.20

Deployment
VantageCloud
VantageCore
Edition
Enterprise
IntelliFlex
VMware
Product
Analytics Database
Teradata Vantage
Release Number
17.20
Published
June 2022
Language
English (United States)
Last Update
2023-01-27
dita:mapPath
aju1628095815656.ditamap
dita:ditavalPath
qkf1628213546010.ditaval
dita:id
hqj1472245413611

To determine the length of a byte sequence in UTF8, examine the first byte.

IF the... THEN the sequence is...
high order bit is zero one byte long.

This leaves seven bits to encode information.

If a character has a Unicode value that can be represented in seven bits, it is represented as a byte containing the Unicode value. For example, Unicode value 0x0041 is transformed to UTF8 byte 0x41.

three high order bits are 110 a two-byte sequence.

The second byte has the two high order bits set to 10. There are five free bits in the first byte and six free bits in the second byte. This allows eleven bits to represent a numeric value.

If a character has a Unicode value that can be represented in eleven bits, and cannot be represented by a shorter UTF8 sequence, then it is represented as two bytes, where the free bits contain the Unicode value. For example, Unicode value 0x03F1 is transformed to UTF8 byte sequence 0xCF 0xB1.

four high order bits are 1110 a three byte sequence.

The second and third bytes have the two high order bits set to 10. There are four free bits in the first byte and six free bits in each of the second and third bytes. This allows sixteen bits to represent a numeric value.

If a character has a Unicode value that can be represented in sixteen bits, and can not be represented by a shorter UTF8 sequence, it is represented as three bytes, where the free bits contain the Unicode value (for example, Unicode value 0x3000 is transformed to UTF8 byte sequence 0xE3 0x80 0x80).