11

Suppose I have a value that is large in memory (perhaps a huge matrix). Is there a way to move that value to a different environment instead of copy then delete? The copy/clone approach temporarily increases the memory footprint by the size of the value.

I reviewed this post but it doesn't contain the solution to my problem. Sharing the same environment (to avoid the copy) is not an option. I really do need to move the value.

Community
  • 1
  • 1
Suraj
  • 35,905
  • 47
  • 139
  • 250
  • 4
    Can you provide an example of how copying (specifically `for(n in ls(e1, all.names=TRUE)) assign(n, get(n, e1), e2)`) increases the memory footprint? I just tried that and the maximum memory used--as reported by `gc()`--did not change. – Joshua Ulrich Feb 06 '13 at 16:40
  • 1
    I think you're right. It doesn't increase the memory footprint. Perhaps because of promises? It seems that you can assign to new environment and remove from current environment and as long as you don't evaluate you are good. – Suraj Feb 06 '13 at 16:44
  • 2
    A copy will be made if you change an object in the new environment before you've deleted it from the old environment. – Joshua Ulrich Feb 06 '13 at 16:47
  • good debugging! Thanks, Josh! – Suraj Feb 06 '13 at 16:49
  • 1
    @JoshuaUlrich Post your comments as an answer please? – Roland Feb 07 '13 at 08:29
  • @Roland: your wish is my command. :) – Joshua Ulrich Feb 07 '13 at 12:40

1 Answers1

7

Perhaps write to disk, delete, read from disk? The only potential problem I can foresee with this approach is that any relationships between parent/child environments will be lost. But if you're simply trying to copy the values from one environment to another, maybe this isn't a problem?

Update:

I cannot replicate what you say about the copy approach. The code below shows that the maximum memory used (as reported by gc) does not increase. This is because the values are "promised", not deep-copied. A copy will be made, however, if you change an object in the new environment before you delete it from the old environment.

R> e1 <- new.env()
R> e1$x <- numeric(5e7)
R> e1$y <- numeric(5e7)
R> gc()
            used  (Mb) gc trigger  (Mb)  max used  (Mb)
Ncells    171022   9.2     350000  18.7    350000  18.7
Vcells 100271746 765.1  110886821 846.0 100272535 765.1
R> e2 <- new.env()
R> for(n in ls(e1, all.names=TRUE))
+   assign(n, get(n, e1), e2)
R> gc()
            used  (Mb) gc trigger  (Mb)  max used  (Mb)
Ncells    171038   9.2     350000  18.7    350000  18.7
Vcells 100271788 765.1  116511162 889.0 100272535 765.1
R> identical(e1$x,e2$x)
[1] TRUE
R> identical(e1$y,e2$y)
[1] TRUE
Joshua Ulrich
  • 173,410
  • 32
  • 338
  • 418
  • Darn! You posted before I could specify that serialization to disk wasn't an option. I don't need to preserve the parent/child relationships, but the disk solution is a tradeoff between memory and performance. I was wondering if there's an in-memory solution – Suraj Feb 06 '13 at 16:04