Encoding Schemes: ASCII, ISCII and UNICODE (UTF8, UTF32)

By definition, encoding is a method of converting data from one format to another. When we have text (a series of characters) that we wish to keep inside a computer (machine) or transport over a digital network, we must convert it to binary representation since that is the only language that a binary-based computer can comprehend.

Character encoding is a method of converting text input to binary values. In a nutshell, we may give individual characters unique numeric values and transform those numbers to binary. Based on their values, these binary integers may be translated back to original characters.

Unicode

Computers, at their core, only deal with numbers. They keep track of letters and other characters by giving each one a number. Before Unicode, there were hundreds of different ways for allocating these numbers, known as character encodings. These early character encodings were restrictive and could not accommodate all of the world’s languages. Even for a single language like English, no one encoding could accommodate all of the letters, punctuation, and technical symbols in use.

Early character encodings were also incompatible with one another. In other words, two encodings might use the same number for two distinct characters or different numbers for the same character. Any given computer (particularly servers) would need to handle a wide range of encodings. However, when data is transmitted between computers or between various encodings, it is at risk of corruption.

Unicode has altered everything! The Unicode Standard assigns a unique number to each character, regardless of platform, device, application, or language. It has been embraced by all current software vendors and now enables data to be transmitted without corruption across many various platforms, devices, and apps. Unicode support is the basis for language and symbol representation in all major operating systems, search engines, browsers, laptops, and smartphones, as well as the Internet and World Wide Web. The best approach to implement ISO/IEC 10646 is to support Unicode.

Among the most important recent worldwide software technology developments are the creation of the Unicode Standard and the availability of tools to support it.

ASCII

ASCII is an abbreviation for American Standard Code for Information Interchange and is pronounced “ask-ee.” This is the most common alphanumeric code used in computers. It comes in two forms: 7-bit and 8-bit code. 128 characters may be represented using ASCII-7. Three of the seven bits are zone bits, while the remaining four are numeric bits. ASCII-8 can store up to 256 characters. Four of the eight bits are zone bits, while four are numeric bits. It’s an expanded version of ASCII-7. It has all normal keyboard characters as well as control features.

ISCII

ISCII is a coding technique used to represent several Indian writing systems. It encodes the primary Indian scripts as well as a Roman transcription. ISCII standard was accepted by the Bureau of Indian Standards in 1991, the ISCII standard that was developed by a standardisation group, of which C-DAC was a member, under the Department of Electronics between 1986-88. It uses an 8-bit encoding scheme and includes around 256 characters. The first 128 characters, numbered 0-127, are the same as in ASCII.

What is UTF8 Encoding?

UTF-8 is a Unicode encoding system. It can convert any Unicode character to a matching unique binary string and then back to a Unicode character. This is what “UTF,” or “Unicode Transformation Format,” means.

There are various Unicode encoding systems other than UTF-8, however UTF-8 is unique, it represents characters in one-byte units. Remember that a byte is made out of eight bits, thus the “-8” in the name.

UTF-8, in particular, translates a code point (which in Unicode represents a single character) into a collection of one to four bytes. The first 256 characters in the Unicode library, including the ASCII characters, are represented by one byte. Later in the Unicode library, characters are encoded as two-byte, three-byte, and finally four-byte binary units.

Conclusion

Character encoding is the technique of assigning numbers to graphical characters, particularly human language written characters, in order for them to be stored, communicated, and altered using digital computers.  The numerical values that make up a character encoding are known as “code points,” and they form a “code space,” a “code page,” or a “character map” when combined.

Early character codes connected with the optical or electrical telegraph could only represent a fraction of the characters used in written languages, and were occasionally limited to upper case letters, numbers, and certain punctuation. Because of the cheap cost of digital data representation in current computer systems, more complicated character codes (such as Unicode) that represent the majority of the characters used in many written languages are possible. The use of globally established standards for character encoding allows for the worldwide exchange of text in electronic form.