3

As I understand the documentation for the log section of a snakemake rule, one has to "manually" send things to the log files. It seems to me that one could achieve the same results using files defined in the output section.

What are the important differences between these two possible approaches?

What is the real usefulness of the log section?

bli
  • 7,549
  • 7
  • 48
  • 94
  • I'm not sure, but I would say it looks "cleaner", since output files are used by Snakemake to chain the rules, and determine which rules are to be run, depending on which files are present/absent... – rioualen Mar 17 '17 at 09:59
  • A thing I noticed is that all `output` seem to have to include the same set of wildcards. Might one want to have log files without some of these wildards? Likely not, because this would imply overriding logs from other instances of the same rule. – bli Mar 17 '17 at 10:29

1 Answers1

5

For me the best pratice for log is Snakemake is like that :

rule example1:
  input:
    file = <input>

  log: 
    out = '_stdout.log',
    err = '_stderr.err'

  output:
    <output>

  shell: 
    'Script/Tool {input.file} 2> {log.err} 1> {log.out}'

The log section is very useful I think. Most programs or tools produce some logs on standard out and standard error.This is useful for the user to know at which step of the tool or program it fails.

Of course you can do it on the output section like the following code :

rule example2:
  input:
    file = <input>

  output:
    file = <output>
    out = '_stdout.log',
    err = '_stderr.err'

  shell: 
    'Script/Tool {input.file} 2> {output.err} 1> {output.out}'

This will produce the same results as the example1 rule. But the purpose of output section is to make dependencies with other rules or just provide the results files you needed. In most cases, logs aren't these files, unless in a rule to check some parameters or files.

There is one big disadvantage to put the log on output. When a rule in Snakemake fails, Snakemake delete all the output which might be corrupted by the fail. So your log will be deleted too, and you might not be able to see at which step of the program it fails or the reason of it.

Hugo

Pereira Hugo
  • 337
  • 2
  • 7
  • 1
    The deletion of the output files is a really important point. Thanks. – bli Mar 16 '17 at 16:25
  • How would this work had you used `script:` rather than `shell:` ? – anilbey Feb 02 '18 at 12:44
  • I believe @anilbey 's question is covered [here](https://stackoverflow.com/q/64101921/8508004), and a related option [here](https://stackoverflow.com/a/55833804/8508004). – Wayne Jul 01 '22 at 14:52