Rényi Entropy Rate of Stationary Ergodic Processes

Abstract

— In this paper, we examine the Rényi entropy rate of stationary ergodic processes. For a special class of stationary ergodic processes, we prove that the Rényi entropy rate always exists and can be approximated by its defining sequence at most polynomially; moreover, using the Markov approximation method, we show that the Rényi entropy rate can be exponentially approximated by that of the Markov approximating sequence, as the Markov order goes to infinity. For the general case, by constructing a counterexample, we disprove the conjecture that the Rényi entropy rate of a general stationary ergodic process always converges to its Shannon entropy rate as α goes to 1.