how to adjust future.global.maxSize in R?

r future options
Roger-123 picture Roger-123 · Nov 10, 2016 · Viewed 8.3k times · Source

Can anyone help me figure out how to change the max size of global objects passed to an item in the future package?

Here's a useless example that shows my point

library(future)
a = 1:200000000
object.size(a)
test %<-% head(a)

I get the following error:

Error in getGlobalsAndPackages(expr, envir = envir, persistent = persistent, : The total size of all global objects that need to be exported for the future expression (‘head(a)’) is 762.95 MiB. This exceeds the maximum allowed size of 500.00 MiB (option 'future.global.maxSize'). There are two globals: ‘a’ (762.94 MiB of class ‘numeric’) and ‘head’ (10.05 KiB of class ‘function’).

Can anyone help me understand how to adjust that future.global.maxSize option? I tried options(future.global.maxSize = 1500000) but that didn't work.

Answer

Roger-123 picture Roger-123 · Nov 12, 2016

Got it figured out and learned how you can edit options for any package.

This is the line that I used (edit: the change was from 'global' to 'globals':

options(future.globals.maxSize= 891289600)

If you want to customize your limit, I saw in the package source that the limit was calculated and this is how you would calculate the size for an 850mb limit:

850*1024^2 = 891289600

Thanks!

Related questions

Remove rows with all or some NAs (missing values) in data.frame

I'd like to remove the lines in this data frame that: a) contain NAs across all columns. Below is my example data frame. gene hsap mmul mmus rnor cfam 1 ENSG00000208234 0 NA NA NA NA 2 ENSG00000199674 0 2 2 2 2 3 ENSG00000221622 0 NA NA NA NA 4 …

Read More
Drop data frame columns by name

I have a number of columns that I would like to remove from a data frame. I know that we can delete them individually using something like: df$x <- NULL But I was hoping to do this with …

Read More
Changing column names of a data frame

I have a data frame called "newprice" (see below) and I want to change the column names in my program in R. > newprice Chang. Chang. Chang. 1 100 36 136 2 120 -33 87 3 150 14 164 In fact this is what am doing: names(newprice)[1]<-paste("premium") …

Read More