Analogy between MSE estimation theory and information theory in the study of tandem communication links

Abstract

This research deals with developing an analogy between concepts in information theory and estimation theory in tandem communication problems. The analogy proposed by Messerschmitt [1] provides the intuitive basis for establishing a correspondence between the concept of entropy/mutual information of information theory and mean squared error (MSE) estimation theory with respect to an N-link discretetime communication system. This research analyzes and develops mathematically this correspondence and identifies the conditions in which this correspondence holds. An investigation of the simple 1-1 ink and 2-link communication systems is accomplished as a preliminary step. Based on the results of this investigation, generalized results of the simple 1-link and 2-link communication systems to the N-link communication system are performed. The study of the discrete-time communication system is achieved by considering two contexts: the channel coding context and the source coding context. Continuous random variables are considered with a special focus on the Gaussian case.