Asymptotic Distortion Behavior of the Distributed Karhunen-Loève Transform

Author

Olivier Roy

Abstract

The distributed Karhunen-Loèe Transform (dKLT) is the distributed counterpart to the well known KLT which has received considerable attention over the past decades. The KLT plays a pivotal role in transform coding and the dKLT is expected to be central in solving distributed approximation problems such as the ones encountered in sensors networks. Therefore, a careful analysis of the distortion is needed in order to asses the performance of the transform and, in particular, the loss incurred with respect to the centralized scenario. In this project, we consider the mean squared distortion behavior as the size of the source vector becomes large.

We study the distortion from a pure approximation point of view by defining the asymptotic normalized distortion of the source reconstruction. Extending some results about large Toeplitz matrices, we show how to compute the asymptotic normalized distortion of stationary processes for various scenarios of both theoretical and practical interest. This allows us to precisely compare our approach to the related rate-distortion (compression) framework usually considered from an information theoretic point of view. In particular, we show how the distortions from both perspectives compare.
In a second part, we apply our results to a first-order Gauss-Markov process and provide, for both the approximation and the compression standpoint, closed-form formulas of the distortion. This gives significant insights about the performance of the dKLT. For the particular case at hand, we are able to precisely quantify the gain due to the presence of side information at the decoder or the loss incurred by the need to interpolate missing data. Finally, we also show how the considered scenarios relate to the more general setup whose precise characterization is still unknown to date.

Based on this work, our focus now is to study an information theoretic model for hearing aids. We are interested to analysis the impact of the transmission rate on the estimation of critical parameters needed for accurate beamforming. This approach is motivated by the practical setup where hearing aids have to exchange information, possibly in an interactive manner, while keeping a low transmission power due to energy constraints.

Major Publications

 

O. Roy and M. Vetterli, On the Asymptotic Distortion Behavior of the Distributed Karhunen-Loève Transform, Allerton Conference on Communication, Control and Computing, pp. 406-415, 2005.
[detailed record] [bibtex]

O. Roy and M. Vetterli, Dimensionality Reduction for Distributed Estimation in the Infinite Dimensional Regime, IEEE Transactions on Information Theory, Vol. 54, Nr. 2, pp. 1655-1669, 2008.
[detailed record] [bibtex]

Collaborations

Martin Vetterli

Funding

National Competence Center in Research on Mobile Information and Communication Systems (NCCR-MICS), a center supported by the Swiss National Science Foundation under grant number 5005-67322.

Author

Olivier Roy

Abstract

The distributed Karhunen-Loèe Transform (dKLT) is the distributed counterpart to the well known KLT which has received considerable attention over the past decades. The KLT plays a pivotal role in transform coding and the dKLT is expected to be central in solving distributed approximation problems such as the ones encountered in sensors networks. Therefore, a careful analysis of the distortion is needed in order to asses the performance of the transform and, in particular, the loss incurred with respect to the centralized scenario. In this project, we consider the mean squared distortion behavior as the size of the source vector becomes large.

We study the distortion from a pure approximation point of view by defining the asymptotic normalized distortion of the source reconstruction. Extending some results about large Toeplitz matrices, we show how to compute the asymptotic normalized distortion of stationary processes for various scenarios of both theoretical and practical interest. This allows us to precisely compare our approach to the related rate-distortion (compression) framework usually considered from an information theoretic point of view. In particular, we show how the distortions from both perspectives compare.
In a second part, we apply our results to a first-order Gauss-Markov process and provide, for both the approximation and the compression standpoint, closed-form formulas of the distortion. This gives significant insights about the performance of the dKLT. For the particular case at hand, we are able to precisely quantify the gain due to the presence of side information at the decoder or the loss incurred by the need to interpolate missing data. Finally, we also show how the considered scenarios relate to the more general setup whose precise characterization is still unknown to date.

Based on this work, our focus now is to study an information theoretic model for hearing aids. We are interested to analysis the impact of the transmission rate on the estimation of critical parameters needed for accurate beamforming. This approach is motivated by the practical setup where hearing aids have to exchange information, possibly in an interactive manner, while keeping a low transmission power due to energy constraints.

Major Publications

 

O. Roy and M. Vetterli, On the Asymptotic Distortion Behavior of the Distributed Karhunen-Loève Transform, Allerton Conference on Communication, Control and Computing, pp. 406-415, 2005.
[detailed record] [bibtex]

O. Roy and M. Vetterli, Dimensionality Reduction for Distributed Estimation in the Infinite Dimensional Regime, IEEE Transactions on Information Theory, Vol. 54, Nr. 2, pp. 1655-1669, 2008.
[detailed record] [bibtex]

Collaborations

Martin Vetterli

Funding

National Competence Center in Research on Mobile Information and Communication Systems (NCCR-MICS), a center supported by the Swiss National Science Foundation under grant number 5005-67322.