Nyquist and Information Theory

Information theory is often considered to have begun with work by Harry Nyquist (H. Nyquist, Certain factors affecting telegraph speed, Bell System Technical Journal, 3, 324-346, 1924). While new knowledge is built by individuals standing on the shoulders of those who performed earlier research, people such as Nyquist can be seen as being extraordinarily creative for putting together previous work to produce a new and unique model.

Writing in the Bell System Technical Journal, Nyquist suggested that two factors determine the "maximum speed of transmission of intelligence". Each telephone cable is implicitly considered to have a limit imposed on it such that there is a finite, maximum speed for transmitting "intelligence". This limit was widely understood by practicing electrical engineers of the era to be related to such factors as power, noise, and the frequency of the intelligent signal5. Accepting such a limit as a given, Nyquist was able to work backwards towards the study of what was transmitted. He began referring to what was transmitted as "information."

The two fundamental factors governing the maximum speed of data transmission are the shape of a signal and the choice of code used to represent the intelligence. Responding to the earlier work of Squier and others, Nyquist argues that telegraph signals are most efficiently transmitted when the intelligence carrying waves are rectangular. Given a particular "code", use of square waves allows for intelligence to be transmitted faster than with sine waves in many practical environments.

Once the proper wave form is selected, a different problem arises: how should "intelligence" be represented? Telegraphers had long used Morse code and its variants to transmit text messages across distances. Each character was represented by a set of short or long electronic signals, the familiar dots and dashes. The letter C, for example, is represented in modern Morse code by a dash dot dash dot sequence. Experienced telegraphers listen to messages at speeds far exceeding the ability of humans to consciously translate each individual dash or dot into a "thought representation" of the symbol; instead, Morse code is heard as a rhythm, with the rhythm for letters and common words being learned through long periods of listening 6.

Working backwards from the maximum telegraph speed, Nyquist considered the characteristics of an "ideal" code. Morse code is adequate for many applications, but an "adequate code" is far from being the best or optimal code available. Suggesting that the speed of intelligence transmission is proportional to the logarithm of the number of symbols which need to be represented, Nyquist was able to measure the amount of intelligence that can be transmitted using an ideal code. This is one step away from stating that there is a given amount of intelligence in a representation.

After his retirement, Nyquist was employed as a part time consultant engineer on communication matters by the Department of Defense, Stavid Engineering Inc., and the W.L. Maxson Corporation.

Before his death in 1976 Nyquist received many honors for his outstanding work in communications. He was the fourth person to receive the National Academy of Engineer's Founder's Medal, "in recognition of his many fundamental contributions to engineering." In 1960, he received and the IRE Medal of Honor “for fundamental contributions to a quantitative understanding of thermal noise, data transmission and negative feedback." Nyquist was also awarded the Stuart Ballantine Medal of the Franklin Institute in 1960, and the Mervin J. Kelly award in 1961. He passed away on 4 April 1976.

Task I

Tell about Nyquist’s investigations and inventions.

Task II

Speak on Nyquist Theorem.

Task III

Discuss Harry Nyquist’s contribution to Information Theory.

Date: 2015-12-24; view: 932