Calculation of mutual information in R

r entropy information-theory
lemhop picture lemhop · Sep 11, 2014 · Viewed 12.1k times · Source

I am having problems interpreting the results of the mi.plugin() (or mi.empirical()) function from the entropy package. As far as I understand, an MI=0 tells you that the two variables that you are comparing are completely independent; and as MI increases, the association between the two variables is increasingly non-random.

Why, then, do I get a value of 0 when running the following in R (using the {entropy} package):

mi.plugin( rbind( c(1, 2, 3), c(1, 2, 3) ) )

when I'm comparing two vectors that are exactly the same?

I assume my confusion is based on a theoretical misunderstanding on my part, can someone tell me where I've gone wrong?

Thanks in advance.

Answer

Monicam picture Monicam · Jul 15, 2016

Use mutinformation(x,y) from package infotheo.

> mutinformation(c(1, 2, 3), c(1, 2, 3) ) 
[1] 1.098612

> mutinformation(seq(1:5),seq(1:5))
[1] 1.609438

and normalized mutual information will be 1.

Related questions

Calculating Entropy

I've tried for several hours to calculate the Entropy and I know I'm missing something. Hopefully someone here can give me an idea! EDIT: I think my formula is wrong! CODE: info <- function(CLASS.FREQ){ freq.class <…

Read More
Remove rows with all or some NAs (missing values) in data.frame

I'd like to remove the lines in this data frame that: a) contain NAs across all columns. Below is my example data frame. gene hsap mmul mmus rnor cfam 1 ENSG00000208234 0 NA NA NA NA 2 ENSG00000199674 0 2 2 2 2 3 ENSG00000221622 0 NA NA NA NA 4 …

Read More
Drop data frame columns by name

I have a number of columns that I would like to remove from a data frame. I know that we can delete them individually using something like: df$x <- NULL But I was hoping to do this with …

Read More