Computing & Digital Codexery

Character encoding

Last updated

A convention for representing writing script characters through numeric values is what character encoding refers to.[1] Code points, the numerical values in an encoding, enable computer storage, transmission, and transformation of character data.[1] Among web sites whose encoding is known, UTF-8 holds a 98.9% usage share, making it the dominant World Wide Web character encoding.[2]

Key facts and history

IBM used six-bit binary-coded decimal character encoding schemes starting as early as 1953 in its 702 and 704 computers.[3] In 1963 the first ASCII code was released, using a simpler seven-bit code that addressed most shortcomings of the earlier Fieldata code.[4] ASCII67 was adopted fairly widely after adding lower-case letters and fixing control code issues.[4] In the 1980s, researchers faced the dilemma that adding more bits to accommodate additional characters seemed necessary, but those bits were a waste for users of the Latin alphabet.[5]

Unicode and the ISO/IEC 10646 Universal Character Set together constitute a unified standard for character encoding.[6] Unicode separately defines a coded character set mapping characters to code points, how code points map to code units, and how units encode as byte streams.[6] The Unicode Standard defines three standard encoding forms: UTF-8, UTF-16, and UTF-32.[7] Morse code, which first appeared in the 1840s as an electrically transmitted character code, employed four symbols (short signal, long signal, short space, long space) to create variable-length codes.[8]

All ASCII characters plus extra non-ASCII ones were supported by eight-bit extended ASCII encodings, for instance the ISO/IEC 8859 series and various vendor-specific extensions.[4]

What the cited sources add

The Unicode Character Encoding Model places the Unicode Standard in the context of other character encodings and models such as IBM's Character Data Representation Architecture.[9] The Unicode model gives the transfer encoding syntax a separate status outside the character encoding model proper.[9] The synchronized repertoire of Unicode and 10646 is intended to be universal in coverage, unlike most character repertoires.[9] The identity of abstract characters in other coded character sets is increasingly defined by reference to Unicode and 10646.[9]

The Tech Terms Dictionary notes that while ASCII is still supported by nearly all text editors, Unicode is more commonly used because it supports a larger character set.[10] The Android Developers documentation defines a charset as the combination of one or more coded character sets and a character-encoding scheme.[11] The Java platform's native character encoding is UTF-16, and a charset defines a mapping between sequences of UTF-16 code units and sequences of bytes.[11]

References

  1. Wikipedia, "Character encoding" (CC BY-SA 4.0), which cites: "Character Encoding Definition", The Tech Terms Dictionary, September 24, 2010 (we also read that source) en.wikipedia.org
  2. Wikipedia, "Character encoding" (CC BY-SA 4.0), which cites: "Usage Survey of Character Encodings broken down by Ranking", W3Techs; "Charset", Android Developers (we also read that source) en.wikipedia.org
  3. Wikipedia, "Character encoding" (CC BY-SA 4.0), which cites: "IBM Electronic Data-Processing Machines Type 702 Preliminary Manual of Information", 1954, p. 80 (original not read by us) en.wikipedia.org
  4. Wikipedia, "Character encoding" (CC BY-SA 4.0), which cites: Tom Jennings, "An annotated history of some character codes", Sensitive Research, 20 April 2016 (original not read by us) en.wikipedia.org
  5. Wikipedia, "Character encoding" (CC BY-SA 4.0), which cites: Strelho, "IBM Drives Hard Disks to New Standards", InfoWorld, April 15, 1985, p. 29–33 (original not read by us) en.wikipedia.org
  6. Wikipedia, "Character encoding" (CC BY-SA 4.0), which cites: Whistler, "UTR#17: Unicode Character Encoding Model", Unicode Consortium, 2022-11-11 (we also read that source) en.wikipedia.org
  7. "Glossary of Unicode Terms", Unicode Consortium. facts only, no text reused. unicode.org
  8. Wikipedia, "Character encoding" (CC BY-SA 4.0), which cites: Tom Henderson, "Ancient Computer Character Code Tables – and Why They're Still Relevant", Smartbear, 17 April 2014 (original not read by us) en.wikipedia.org
  9. Whistler, "UTR#17: Unicode Character Encoding Model", Unicode Consortium, 2022-11-11. facts only, no text reused. unicode.org
  10. "Character Encoding Definition", The Tech Terms Dictionary, September 24, 2010. facts only, no text reused. techterms.com
  11. "Charset", Android Developers. facts only, no text reused. developer.android.com

Each numbered sentence above was checked against the source it cites before publication. Sentences citing Wikipedia adapt text from the English Wikipedia article Character encoding (contributors), used under CC BY-SA 4.0; this entry is available under the same licence. Other sources were used for facts only; no text or images are copied from them.

More in Computing & Digital

Sources

Compiled from Wikipedia and the sources listed below. Text from Wikipedia is available under CC BY-SA 4.0; this entry is adapted from it.

Spotted an error? Know more?

Reader corrections go straight into our review queue. Suggest an edit · How this site is sourced

Comments

Loading…
Open in the interactive codex →