Communication, Codes and Cyphers

2.1 Instantaneously Decipherable Codes

Most of the codes that we are dealing with in this course are not only uniquely decipherable, but are what we call instantaneously decipherable. These are codes that we are able to decipher as we go along - this is an important property for codes used in communication, since we don't want to have to wait for the whole message to arrive before we work out what it says.

For simplicity, we will assume that instantaneously decipherable codes will be built by mapping individual characters from the source alphabet into words in the code alphabet, or codewords. The encoded form of the message is then just built up by concatenating the code words together. This is nothing surprising, since this is the way that Morse code works. It is also not a great restriction, since if we wanted to work with mappings of multiple blocks of characters, we can replace our source alphabet with the alphabet whose characters are those blocks.

Example

Consider the code with source alphabet {A,B,C,D,E}, code alphabet {0,1} and codewords:

Source	A	B	C	D	E
Code	0	1001	110	101	111

The message "ACE" would be encoded into the message "0110111".

Example

ISBNs do not immediately seem to fall into this scheme, since we construct them by taking a sequence of 9 digits of the alphabet {0,1,...,9} into a sequence of 10 digits of the same alphabet. However, if we instead think of the sequence of 9 digits as being one character of a huge alphabet, then we can think of the ISBN code as a code which is built up out of mappings of characters to codewords.

By an instantaneously decipherable code, we mean one that we can immediately tell when we have reached the end of the codeword for a character in the source alphabet. In other words, no codeword for a letter can be the start of the codeword for any other letter.

Example

Morse code, where we add the additional code character "/" to mark the end of codewords, is uniquely decipherable.

We could think of another way of making Morse code uniquely decipherable by appending a list of numbers at the end which tell you how many characters were in each code word. For example, SOS might become

· · · - - - · · · 3 - 3 - 3 -

which tells you that the first letter is the first 3 characters, the second is the second 3, and the third is the third block of 3. By contrast VMS would be encoded as

· · · - - - · · · 4 - 2 - 3 -

This version of Morse code is not uniquely decipherable, since we have to wait to the end of the message to work out what the message is.

We could also change Morse code by putting a "/" at the start of each codeword, so that SOS would be encoded as

/ · · · / - - - / · · ·

This code is uniquely decodable, but is not instantaneously decipherable, since we have that the codeword for E (/ ·) is the start of the codeword for S (/ · · ·).

A simple way to guarantee that your code is instantaneously decipherable is to ensure that all your codewords have the same length. This is the approach taken by the ASCII code - every letter in the source alphabet is converted into a sequence of 7 binary digits. This approach is usually good for encryption and error-correcting codes, but it is far from optimal when compressing, since it does not allow more frequently used letters to have shorter codewords.