|
|
Computing the Entropy Rate of Information Source with Methods of Statistical Physics |
Chen Shuang-ping①; Zheng Hao-ran②; Ma Meng②③; Zhang Zhen-ya①; Wang Xu-fa② |
①Department of Electronic Engineering and Information Science, University of Science and Technology of China (USTC), Hefei 230027, China; ②Department of Computer Science and Technology, USTC, Hefei 230027, China; ③School of Computer Science and Technology, Anhui University, Hefei 230039, China |
|
|
Abstract From the mathematical point of view, information sources can be 1-to-1 mapped to stochastic processes. Known from the theory of chaos, multi-fractal of stochastic process is a key characteristic of its dynamics, of which entropy rate is a special fractal dimension named information dimension. The paper introduces methods of statistical physics to compute the multi-fractal of stochastic process so that the entropy rate of source can be obtained at once. Take binary hidden Markov processes as example, the paper demonstrate how this approach works. The results shows that the methods is applicable to numerically approximate the entropy rate of binary hidden Markov processes (BHMPs) in practical applications, and it can be applied in more generalized kinds of information sources.
|
Received: 23 May 2005
|
|
|
|
|
|
|
|