Exporting g++ compilation error messages from TMB library to a text file

Question

I'm trying to compile a model with gcc (c++) in R (using the TMB package). The errors are so numerous that in Rstudio, I can't even scroll up to see the beginning of them. Therefore, I would like to print everything in the console (messages, errors and warnings) to a text file. This way I could also compare different model outputs (or specifically, their fails).

I tried the following things:

fileConn<-file("Fail.txt")
writeLines(compile("Mymodel.cpp"), fileConn) 
close(fileConn)

=> Gives me an empty file (which I more or less expected)

zz <- file("Fail.txt", open = "wt")
sink(zz, type = "message")
compile("Mymodel.cpp")
sink()

=> only prints the final error

What did I overlook?

Add `sink(zz, type = "output")` after the first `sink`. In this way you divert both `stdout` and `stderr` to the file you choose. — nicola, Mar 07 '16 at 16:54
Are you sure? Please, provide a reproducible example, with expected output. — nicola, Mar 07 '16 at 17:07

score 2 · Accepted Answer · edited May 23 '17 at 11:50

You haven't provided your compile() function, but I assume it runs a system command that calls g++ to compile Mymodel.cpp.

In this case, the g++ process will print its error output to stderr. The only way to capture this output using R is to call system2() with stderr=T. Note that system() does not have the capability to capture stderr directly (although it can capture stdout via intern=T, and you could add 2>&1 to the shell command to capture stderr along with it, so that's another viable option). sink() does not capture system command output; it only captures R output.

I recommend passing variadic arguments from your compile() function to the system2() call, thus parameterizing whether your call to compile() results in g++'s stderr going to the terminal or to the return value of your compile() function. Here's how this would be done:

write('error!','test1.cpp'); ## generate a test file with invalid C++
compile <- function(file,...) system2('g++',file,...);
compile('test1.cpp'); ## output lost to the terminal
## test1.cpp:1:1: error: ‘error’ does not name a type
##  error!
##  ^
output <- compile('test1.cpp',stdout=T,stderr=T); ## capture output
## Warning message:
## running command ''g++' 'test1.cpp' 2>&1' had status 1
output;
## [1] "test1.cpp:1:1: error: ‘error’ does not name a type"
## [2] " error!"
## [3] " ^"
## attr(,"status")
## [1] 1
write(output,'output.txt'); ## write output to a text file
cat(readLines('output.txt'),sep='\n'); ## show it
## test1.cpp:1:1: error: ‘error’ does not name a type
##  error!
##  ^

If you really want to capture all output generated within your compile() function, then you can combine the above solution with sink() as demonstrated here: How to save all console output to file in R?.

In this case, I'd recommend scrapping the variadic arguments idea and going with a single additional argument to compile(), which will take an output file name to which all output will be written.

This will require several predications on the missingness of the additional argument:

write('error!','test1.cpp'); ## generate a test file with invalid C++
compile <- function(file,outputFile) {
    if (!missing(outputFile)) {
        outputCon <- file(outputFile,'wt'); ## require file name
        sink(outputCon);
        sink(outputCon,type='message'); ## must sink messages separately
        warn.old <- options(warn=1)$warn; ## necessary to capture warnings as they occur
    }; ## end if
    cat('some random output 1\n');
    if (!missing(outputFile)) {
        output <- system2('g++',file,stdout=T,stderr=T); ## before flush to get warnings
        sink(); ## force flush before appending system command output
        sink(type='message');
        outputCon <- file(outputFile,'at'); ## must reopen connection for appending
        write(output,outputCon);
        sink(outputCon);
        sink(outputCon,type='message');
    } else {
        system2('g++',file);
    }; ## end if
    cat('some random output 2\n');
    if (!missing(outputFile)) {
        sink();
        sink(type='message');
        options(warn=warn.old);
    }; ## end if
}; ## end compile()
compile('test1.cpp'); ## output lost to the terminal
## some random output 1
## test1.cpp:1:1: error: ‘error’ does not name a type
##  error!
##  ^
## some random output 2
compile('test1.cpp','output.txt'); ## internally capture all output
cat(readLines('output.txt'),sep='\n'); ## show it
## some random output 1
## Warning: running command ''g++' test1.cpp 2>&1' had status 1
## test1.cpp:1:1: error: ‘error’ does not name a type
##  error!
##  ^
## some random output 2

Thank you, I learned something new. However, when I do this, it only creates an empty file again. I also tried system(test1.dpp, intern=T) and system2("test1.cpp", stdout=TRUE,stdout=TRUE) as well, but neither seem to give me the wanted results (only function (x) UseMethod("t") ). I'm sorry for gorgetting to mention that compile() is from the TMB package (and doing what you presumes it does) — Wave, Mar 07 '16 at 17:57
In the `system()` and `system2()` calls you've pasted into your comment, you have incorrect shell command syntax; `g++` must be the first word in the command. Also, in your `system2()` call, you've specified `stdout=T` twice, rather than `stdout=T,stderr=T`. Also, the output you pasted looks like the output you get when you run `t;`, IOW the default dump of the base R `t()` function, which is not relevant to this question. Finally, I looked into `TMB::compile()`, and it provides no facility for capturing its system command output, so you can't achieve this requirement with `TMB::compile()`. — bgoldst, Mar 07 '16 at 19:31
Ok, thank you! And oops, I completely screwed up the copy pasting/retyping. Sorry :s — Wave, Mar 08 '16 at 18:52

Sébastien Rochette · Answer 2 · 2017-07-07T13:54:45.753

1

This is an old question but I found a simple solution to get a logfile output for TMB::compile, which I have not found anywhere else.
As stated above, sink() does not capture system command output; it only captures R output. However, any argument not listed in the function compile from library TMB are passed as Makeconf variables. Thus, you only have to pass a classical output command with the path to the targeted logfile in a string format:

TMB::compile(file = "Mymodel.cpp", "&> /tmp/logfile.log")

edited Jul 07 '17 at 13:54

answered Jul 07 '17 at 13:48

Sébastien Rochette

6,536
2
22
43

a MUCH more elegant approach than the accepted answer! – RTbecard Jun 09 '19 at 10:22

Exporting g++ compilation error messages from TMB library to a text file

2 Answers2