Is there something in testthat like expect_no_warnings()?

Question

I'm writing tests for a function that under some conditions will generate warnings. I want to ensure that under the other conditions it does not produce warnings. I don't see an obvious way to easily test that with testthat. I guess I could do something like:

my.result <- 25
my.func <- function() my.result
expect_equal(
  withCallingHandlers(
    my.func(), warning=function() stop("A Warning!")
  ), 
  my.result
)

or use options(warn=2), but I was hoping there would be something like:

expect_no_warnings(my.func())

Am I missing something obvious?

@BenBolker, I have two issues with this approach, 1. (also a problem with the other alternate I suggested), `testthat` stops execution on error, instead of continuing and reporting any other failed tests; 2. I can't think of an easy systematic mechanism of undoing this that will survive non-handled errors in a `test_that` block. Normally I would do `on.exit(options(warn=old.warn))` but to do this I'd have to wrap a function around the `test_that` block. Adding `options(warn=old.warn)` at the end wouldn't get run with the warning failure and my options would be left changed. — BrodieG, Feb 25 '14 at 17:08

score 34 · Accepted Answer · answered Nov 10 '15 at 20:26

34

In even more recent versions of ´testthat´ (from 0.11.0) you can do:

expect_warning(my.func(), regexp = NA)

From the documentation of expect_error

regexp: regular expression to test against. If omitted, just asserts that code produces some output, messsage, warning or error. Alternatively, you can specify NA to indicate that there should be no output, messages, warnings or errors.

So in the same way you can test that there are no messages, errors and output.

answered Nov 10 '15 at 20:26

alko989

7,688
5
39
62

3

In testthat 0.11.0 you can also use `expect_silent(...)` if you want to check that it emits no output, messages, or warnings all in one step. – Stuart R. Jefferys Feb 29 '16 at 17:21
@StuartR.Jefferys you are right if you want to test all of the output. Nevertheless, the question is about how to expect no warnings. So a test that prints a message will fail, even if no warning was raised. – alko989 Feb 29 '16 at 18:56
1

Absolutely. That's why I added it as a comment and listed the things it catches all in one go. The question did not say *only* warnings should be caught ;) I almost always use expect_silent as it is often what is really being tested and is a more restrictive. Your answer is the best way to test only for a warning and allow everything else. – Stuart R. Jefferys Feb 29 '16 at 19:44
I desire to verify no warning, however I do get an output. ie my function under test always generates a warning. Sometimes it generates a warning. I would like two test cases. One for a warning and one for output but no warning. – netskink Aug 31 '17 at 22:12

score 12 · Answer 2 · answered Feb 26 '14 at 09:16

12

In recent versions of testthat, you can simply do:

expect_that(my.func(), not(gives_warning()))

answered Feb 26 '14 at 09:16

Karl Forner

4,175
25
32

3

As of testthat 0.10.0 (May 22 2015), the `expect_that(A, func(B))` style of tests are soft deprecated, and documentation of `not()` states that this is to be removed in a future release. Given that, is there another answer to this other than setting `options(warning=2)`? – Stuart R. Jefferys Oct 09 '15 at 18:28
Sorry Karl, changed accepted answer to alko's given the deprecation. More details on [github](https://github.com/hadley/testthat/issues/219). – BrodieG Mar 17 '16 at 13:15
actually now the optimal answer is even `expect_warning(my.func(), NA)`, shorter and more clear – Karl Forner Jun 20 '23 at 16:36

score 4 · Answer 3 · edited May 23 '17 at 11:46

4

Update: Use expect_silent() nowadays because expect_that is deprecated, see help for the function!.

Update 2: As mentioned by @eaglefreeman the answer using expect_warning with the param regexp set to NA is the best solution since my answer causes the test to fail even if no warning was raised but just a a message was printed. This is not what the OP wanted (but just ignore warnings). I do not delete this answer to make this difference clear for other readers.

From the help's examples:

expect_silent("123")

f <- function() {
  message("Hi!")
  warning("Hey!!")
  print("OY!!!")
}

expect_silent(f())

Warning: expect_silent also expects no output so the semantics is a little bit different!

edited May 23 '17 at 11:46

Community

1
1

answered Nov 13 '16 at 20:17

R Yoda

8,358
2
50
87

Thanks, since this is a little more general I'm leaving the currently accepted answer as accepted (e.g. I might want to allow stdout, but not warnings for example), but this is good to know nonetheless (+1) – BrodieG Nov 14 '16 at 13:32
I had to downvote this sorry. expect_silent is actually very misleading in this case. – eaglefreeman Mar 28 '17 at 16:04
@eaglefreeman Can you elaborate this please and propose a better answer to help us all to improve this? – R Yoda Mar 29 '17 at 06:28
2

I used expect_silent and lost some times figuring out why it didn't work. I don't want other people to experiment the same thing :) The best answer is expect_warning with regexp set to NA. – eaglefreeman Mar 30 '17 at 06:46
@eaglefreeman THX a lot for your explanation, your are absolutely right! I have updates my answer to warn other readers (instead of deleting it). – R Yoda Mar 30 '17 at 21:10

score 1 · Answer 4 · answered Sep 21 '17 at 14:35

1

You can check whether expect_warning() fails. The following solves both the issue of other kind of output being produced by my.func() and not() being deprecated:

expect_failure(expect_warning(my.func()))

answered Sep 21 '17 at 14:35

Francesco Napolitano

1,538
1
11
8

Is there something in testthat like expect_no_warnings()?

4 Answers4