Shannon's Entropy calculation

matlab statistics probability information-theory
Composer picture Composer · Feb 27, 2014 · Viewed 15.6k times · Source

I have a probability distribution that defines the probability of occurrence of n possible states.

I would like to calculate the value of Shannon's entropy, in bits, of the given probability distribution.

Can I use wentropy(x,'shannon') to get the value and if so where can I define the number of possible states a system has?

Answer

chappjc picture chappjc · Feb 27, 2014

Since you already have the probability distribution, call it p, you can do the following formula for Shannon Entropy instead of using wentropy:

H = sum(-(p(p>0).*(log2(p(p>0)))));

This gives the entropy H in bits.

p must sum to 1.

Related questions

Test if a data distribution follows a Gaussian distribution in MATLAB

I have some data points and their mean point. I need to find whether those data points (with that mean) follows a Gaussian distribution. Is there a function in MATLAB which can do that kind of a test? Or do …

Read More
How can I display empirical pdf of my 100x1 vector data in Matlab?

I have a data which is 100x1 vector. How can I display its empirical pdf in Matlab? Also, if I want to compare the pdf of three vectors on the same graph, then how to do that? Right now I …

Read More
Creating Gaussian random variable with MATLAB

By using randn function I want to create a Gaussian random variable X such that X ~ N(2,4) and plot this simulated PDF together with theoretic curve.

Read More