9

I submitted a package to CRAN and they asked me to replace \dontrun{} by \donttest{} in the Rd-files and resubmit. I was using \dontrun{} to wrap some examples that are supposed to throw error messages.

After replacing \dontrun{} by \donttest{} I conducted some tests and R CMD check still succeeds but it happens that now both devtools::check() and R CMD check --as-cran fail due to the examples wrapped in \donttest{}:

checking examples with --run-donttest ... ERROR

After some browsing I learned that R 4.0.0 has changed R CMD check --as-cran to run \donttest examples. According to the NEWS of R-devel:

"R CMD check --as-cran now runs \donttest examples (which are run by example()) instead of instructing the tester to do so. This can be temporarily circumvented during development by setting environment variable R_CHECK_DONTTEST_EXAMPLES to a false value."

Since I intend to resubmit the package to CRAN, setting _R_CHECK_DONTTEST_EXAMPLES_ to false locally will not help me.

I also found this recent discussion in a devtools issue where Hadley Wickham states that:

"Generally, now if you don't want to run tests on CRAN \dontrun{} is more likely to work, but using \dontrun{} may cause initial submission to fail."

So now I don't know how to proceed because if I resubmit the package with the required changes I already know it will throw an error in R CMD check --as-cran, and hence it will probably fail CRAN's automatic pretests.

EDIT:

As suggested here I tried if(interactive()){} instead of \dontrun{}. This solution succeeds in R CMD check --as-cran and devtools::check() but I don't think it is the most appropriate way to address this problem since it does not work well with example() (throws an error and does not show the remaining examples). \dontrun{} works better with example() as it prints all the examples but comments out the ones wrapped with \dontrun{}.

Pedro Fonseca
  • 311
  • 1
  • 3
  • 11
  • 2
    I’m voting to close this question because it's more appropriate for the `r-package-devel@r-project.org` mailing list ... it's an administrative question rather than a programming question ... – Ben Bolker Sep 01 '20 at 18:40
  • "I was using \dontrun{} to wrap some examples that are supposed to throw error messages." - maybe just remove these from the examples and instead put these to testthat for unit tests? Or are these examples that throw errors needed for the users to understand the package? – Steffen Moritz Sep 01 '20 at 19:20
  • @SteffenMoritz removing these exemples would indeed fix the errors in R CMD check. But in this package I have some functions that stop the execution with informative error messages depending on certain conditions and I think it would be useful (even thought not essential) to illustrate this. – Pedro Fonseca Sep 01 '20 at 19:41
  • Posted an answer :) What I quite often have already seen is, that people add things to the examples, but comment it out. Might be a good compromise. – Steffen Moritz Sep 01 '20 at 19:54
  • @SteffenMoritz comment out the examples is definitely a good possibility and I will probably end up doing that if nothing better comes up. I'm also considering `if(interactive()){}` instead of `\donttest{}`, as suggested here [here](https://github.com/ThinkR-open/prepare-for-cran) – Pedro Fonseca Sep 01 '20 at 21:15
  • Interesting, might be a good idea. Did not use this before. I guess the downside (in comparison to \donttest{} ) is, that this would be code the user actually sees in the example. Which could be somehow confusing. Or am I wrong here? – Steffen Moritz Sep 01 '20 at 21:23
  • 1
    @SteffenMoritz you are right, the documentation will have `if(interactive())` in the lines of the exemples that i'm trying not to run, and in that sense `\donttest{}` would result in "tidier" documentation since `\donttest{}` is not printed. But I don't think `\donttest{}` is an option here since it will return actual errors when running `R CMD check --as-cran` (and also when running `example()`. But this is still an improvement when compared to `\dontrun` because `\dontrun` is also printed to the documentation. – Pedro Fonseca Sep 01 '20 at 21:41
  • @SteffenMoritz Also, `if(interactive()){}` allows the user to run examples line by line since the condition evaluates to `TRUE`. `\donttest{}` is printed to the documentation and so the user has to select only the code that is inside the `{}`, which is not so user friendly – Pedro Fonseca Sep 01 '20 at 21:42

2 Answers2

3

I don't think the package examples are the right place for "examples that are supposed to throw error messages".

Your problem would be easily solved when you move these 'examples' to testthat unit tests.

There is

expect_error()
expect_warning()

to see if your package throws a warning/error as expected.

If you really want to inform users about what they should avoid to input, maybe you can just add it as a comment to the examples or into the other documentation (details, param)

What you see quite regularly in other package examples is the following:

## Example for working
function(x, abc = "5)

## This would give an error because
# function(x, abc = "falsch")

## Working example 2
function(x)
x <- x+y
Steffen Moritz
  • 7,277
  • 11
  • 36
  • 55
  • Thank you for your answer! I'm already using unit tests to make sure the error messages are as expected (among other things). The error messages I mentioned are not about informing users about what kind of input should be avoided. Imagine an `helper_foo(x, y)` that you use inside `foo(x, y)` to stop the execution of `foo()` in case some conditions regarding `x` and `y` are `TRUE` and inform the user which of the conditions triggered the error. `helper_foo(x, y)` is a function and not just code embedded in `foo()` because it can be used in many other functions with similar input structure. – Pedro Fonseca Sep 01 '20 at 21:09
3

If you know that something will throw an error, you can wrap it in try().

## example of failing code
try(stop("Here is an error"))
Hong Ooi
  • 56,353
  • 13
  • 134
  • 187
  • Thank you for your answer. This seems to be the best solution so far. It succeeds in `R CMD check --as-cran` and properly displays the error messages in `example()` without stopping the execution. Im still considering my options but if this ends up being the actual solution I will come back to accept this answer. – Pedro Fonseca Sep 02 '20 at 00:58
  • I can now confirm that CRAN accepted this solution. – Pedro Fonseca Sep 09 '20 at 10:39