Simple Joint Source/Channel Codes

Author:

Michael Gastpar

Introduction

One of the most important concepts in the design of point-to-point communication systems is the separation principle. It says that an optimal communication system may be designed in two independent steps: First, the source is compressed optimally. Thereafter, an optimal channel code is used to transmit the compressed source without error. This result is of astonishingly general validity in the point-to-point case. Hence, it may seem that the problem of point-to-point communication is solved. Unfortunately, however, this consideration defines optimality only in terms of power and end-to-end distortion while it completely disregards both delay and coding complexity. The goal of this project is the study of system which are optimal in all of those senses. Our investigations were triggered by a well-known example: When a Gaussian source is transmitted over an additive white Gaussian noise channel, uncoded transmission is just as good as the most sophisticated coding scheme — but uncoded transmission minimizes at the same time also delay and complexity, simply because it is not possible to have smaller delay or complexity. This opens the quest for other source/channel pairs for which uncoded transmission achieves optimal performance.

Main contributions

Our main contribution is a closed-form description of such source/channel pairs for the case of discrete-time memoryless systems. More precisely, given the description of the statistical behavior of source and channel, we found explicit formulae for the channel input cost function and the distortion measure that make the overall system information-theoretically optimal. Moreover, we also prove that our formulae are necessary conditions. In other words, if a source/channel pair does not satisfy our formulae, then uncoded transmission is strictly suboptimal. Uncoded transmission is meaningful only when source and channel have the same alphabets. This constraint can be removed by allowing for a symbol-by-symbol mapping at encoder and decoder. We have extended our results to that case as well. Moreover, we applied our findings also to some questions regarding joint source/channel block codes, non-ergodic channels and multi-user communication. Simple joint source/channel codes are even more interesting in the case of networks: there, the separation principle does not apply, and it is not known what the optimal achievable performance is nor how to achieve it. Based on our theory, we have developed optimality results for a certain sensor network topology.

Collaborations

Bixio Rimoldi, Martin Vetterli.

Period

August 1999 – December 2002 (end of PhD thesis).

Funding

ETHZ/EPFL exchange fellowship.

Major publications

M. Gastpar, B. Rimoldi and M. Vetterli, To code or not to code: lossy source-channel communication revisited, IEEE Transactions on Information Theory, Vol. 49, Nr. 5, pp. 1147-1158, 2003.
[detailed record] [bibtex]

M. Gastpar and M. Vetterli, Source-Channel Communication in Sensor Networks, Lecture Notes in Computer Science, Vol. 2634, pp. 162-177, 2003.
[detailed record] [bibtex]