SHANNON ENTROPY ESTIMATION BASED ON HIGH-RATE QUANTIZATION THEORY (ThuPmOR4)

Author(s) : Mattias Nilsson (KTH (Royal Institute of Technology), Sweden) Bastiaan Kleijn (KTH (Royal Institute of Technology), Sweden)

Abstract : In this paper we present a method for estimation of the Shannon differential entropy that accounts for embedded manifolds and is based on high-rate quantization theory. It forms an extension of the classical nearest-neighbor entropy estimator and provides simultaneously an explicit estimate of the manifold dimension. A clear advantage of our method over existing methods is that it estimates the more meaningful Shannon entropy rather than the Renyi entropy. Through experiments we confirm the power and the usefulness of our proposed scheme.

Menu