Communication, Codes and Cyphers

1.1 Shannon's Model of Communication

A line of communication can be thought of as having 5 main components: the source, the encoder, the channel, the decoder, and the sink. A message starts at the source and progresses through each component, until it reaches the receiver. Additionally, the channel may have a source of noise interfering with it, and perhaps an interceptor listening to it. The situation can be pictured as follows:

The message itself is considered as consisting of a sequence of letters from the source alphabet. These characters are emitted one at a time by the source and accepted by the encoder, which encodes them into the channel alphabet. The message is then transmitted along the channel, where it may be altered by noise in the line, or may be spied on by an interceptor. The decoder then receives the transmission and translates them back into the original source alphabet and gives them to the sink.

In many of our examples, the source alphabet will be the standard Roman alphabet of English, often without distinguishing between upper and lower case letters, and with perhaps a few additional punctuation marks, such as SPACE, PERIOD, or COMMA. We will usually assume that the channel uses a binary alphabet, ie. the only letters are 0 and 1. However particular examples, such as the genetic code, may have different alphabets.

The assumption that we are using an alphabet excludes some real-life examples. For instance the connection from an amplifier system to a speaker in a standard stereo system has a wire (the channel) which carries a continuously changing voltage corresponding to the volume of the sound (the source) from the amplifier (the encoder) to the speaker (the decoder) which then produces a sound (the sink). But because the voltage varies continuously, there is no way the infinite number of values it can take could be modeled by a finite, discrete alphabet. Commercial radio transmissions, analog mobile phones, cassette tapes, photographs and vinyl records do not fit the model for the same reason. In general, analog technology fails to meet the requirements, while digital technology does.

Although the names of the components of the model are suggestive of a set-up where the channel is a direct, real-time link between the source and the sink (like a wire or radio transmission), this need not be the case. For example, with a CD, the sounds to be recorded (the source - we assume that they have already been sampled digitally) are coded onto the disc by the manufacturing process (the encoder) which sits around for some indefinite period of time, perhaps getting scratched or damaged (noise), before it is played in a CD player (the decoder) and listened to by a listener (the sink). The channel in this case is the actual CD in the time between when it was created and when it is played.