Collecting p-values within pipe (dplyr)

Question

how are you?

So, I have a dataset that looks like this:

    dirtax_trev indtax_trev lag2_majority pub_exp 
    <dbl>       <dbl>       <dbl>         <dbl>    
    0.1542      0.5186      0             9754
    0.1603      0.4935      0             9260      
    0.1511      0.5222      1             8926     
    0.2016      0.5501      0             9682
    0.6555      0.2862      1             10447

I'm having the following problem. I want to execute a series of t.tests along a dummy variable (lag2_majority), collect the p-value of this tests, and attribute it to a vector, using a pipe.

All variables that I want to run these t-tests are selected below, then I omit NA values for my t.test variable (lag2_majority), and then I try to summarize it with this code:

test <- g %>%
 select(dirtax_trev, indtax_trev, gdpc_ppp, pub_exp, 
 SOC_tot, balance, fdi, debt, polity2, chga_demo, b_gov, social_dem,
 iaep_ufs, gini, pov4, informal, lab, al_ethnic, al_language, al_religion,
 lag_left, lag2_left, majority, lag2_majority, left, system, b_system,
 execrlc, allhouse, numvote, legelec, exelec, pr) %>%
 na.omit(lag2_majority) %>%
 summarise_all(funs(t.test(.[lag2_majority], .[lag2_majority == 1])$p.value))

However, once I run this, the response I get is: Error in summarise_impl(.data, dots): Evaluation error: data are essentially constant., which is confusing since there is a clear difference on means along the dummy variable. The same error appears when I replace the last line of the code indicated above with: summarise_all(funs(t.test(.~lag2_majority)$p.value)).

Alternatively, since all I want to do is: t.test(dirtax_trev~lag2_majority, g)$p.value, for instance, I thought I could do a loop, like this: for (i in vars){ t.test(i~lag2_majority, g)$p.value },

Where vars is an object that contains all variables selected in code indicated above. But once again I get an error message. Specifically, this one: Error in model.frame.default(formula = i ~ lag2_majority, data = g): comprimentos das variáveis diferem (encontradas em 'lag2_majority')

What am I doing wrong?

Best Regards!

Answer 1

Your question is not reproducible, please read this for how you could improve its quality.

My answer has been generalised to be reproducible because I don't have your data and cannot therefore adapt your code directly.

Using a tidy approach I'll produce a data frame of p-values for each variable.

library(tidyr)
library(dplyr)
library(purrr)

mtcars %>%
  select_if(is.numeric) %>%
  map(t.test) %>%
  lapply(`[[`, "p.value") %>%
  as_tibble %>%
  gather(key, p.value)

# # A tibble: 11 x 2
#     key      p.value
#   <chr>        <dbl>
# 1   mpg 1.526151e-18
# 2   cyl 5.048147e-19
# 3  disp 9.189065e-12
# 4    hp 2.794134e-13
# 5  drat 1.377586e-27
# 6    wt 2.257406e-18
# 7  qsec 7.790282e-33
# 8    vs 2.776961e-05
# 9    am 6.632258e-05
# 10 gear 1.066949e-23
# 11 carb 4.590930e-11

update

Thank you for updating your question, note that the value you included in your earlier comment is likely from your original dataset and is still not reproducible here. When I run the code, this is the output.

t.test(dirtax_trev ~ lag2_majority, g)$p.value
# [1] 0.5272474

Please frame your questions in a way that anyone can see the problem in the same way that you do.

To build up the formula you are running through the t.test, I have taken a slightly different approach.

library(magrittr)
library(dplyr)
library(purrr)

g <- tribble(
  ~dirtax_trev, ~indtax_trev, ~lag2_majority, ~pub_exp,
  0.1542, 0.5186, 0, 9754,
  0.1603, 0.4935, 0, 9260,
  0.1511, 0.5222, 1, 8926,
  0.2016, 0.5501, 0, 9682,
  0.6555, 0.2862, 1, 10447
)

dummy <- "lag2_majority"

colnames(g) %>%
  .[. != dummy] %>%          # vector of variables to send through t.test
  paste(., "~", dummy) %>%   # build formula as character
  map(as.formula) %>%        # convert to formula class
  map(t.test, data = g) %$%  # run t.test for each, note the special operator
  tibble(
    data.name = unlist(lapply(., `[[`, "data.name")),
    p.value = unlist(lapply(., `[[`, "p.value"))
  )

# # A tibble: 3 x 2
#                      data.name   p.value
#                          <chr>     <dbl>
# 1 dirtax_trev by lag2_majority 0.5272474
# 2 indtax_trev by lag2_majority 0.5021217
# 3     pub_exp by lag2_majority 0.8998690

If you prefer to drop the dummy variable name from data.name, you could modify its assignment in the tibble with:

data.name = unlist(strsplit(unlist(lapply(., `[[`, "data.name")), paste(" by", dummy)))

N.B. I used the special %$% from magrittr to expose the names from the list of tests to build a data frame. I'm sure there are other ways that may be more elegant, however, I find this form quite easy to reason about.