4

Is there a way to have effect size (such as Cohen's d or the most appropriate) directly using emmeans()?

I cannot find anything for obtaining effect size by using emmeans()

post <- emmeans(fit, pairwise~  favorite.pirate | sex)
emmip(fit, ~ favorite.pirate | sex)
atnplab
  • 107
  • 1
  • 7

2 Answers2

2

There is not a built-in provision for effect-size calculations, but you can cobble one together by defining a custom contrast function that divides each pairwise comparison by a value of sigma:

mypw.emmc = function(..., sigma = 1) {
  result = emmeans:::pairwise.emmc (...)
  for (i in seq_along(result[1, ]))
    result[[i]] = result[[i]] / sigma
  result
}

Here's a test run:

> mypw.emmc(1:3, sigma = 4)
  1 - 2 1 - 3 2 - 3
1  0.25  0.25  0.00
2 -0.25  0.00  0.25
3  0.00 -0.25 -0.25

With your model, the error SD is 9.246 (look at summary(fit); so, ...

> emmeans(fit, mypw ~ sex, sigma = 9.246, name = "effect.size")
NOTE: Results may be misleading due to involvement in interactions
$emmeans
 sex    emmean    SE     df lower.CL upper.CL
 female   63.8 0.434   3.03     62.4     65.2
 male     74.5 0.809  15.82     72.8     76.2
 other    68.8 1.439 187.08     65.9     71.6

Results are averaged over the levels of: favorite.pirate 
Degrees-of-freedom method: kenward-roger 
Confidence level used: 0.95 

$contrasts
 effect.size    estimate     SE  df t.ratio p.value
 female - male    -1.158 0.0996 399 -11.624 <.0001 
 female - other   -0.537 0.1627 888  -3.299 0.0029 
 male - other      0.621 0.1717 981   3.617 0.0009 

Results are averaged over the levels of: favorite.pirate 
Degrees-of-freedom method: kenward-roger 
P value adjustment: tukey method for comparing a family of 3 estimates

Some words of caution though:

  1. The SEs of the effect sizes are misleading because they don't account for the variation in sigma.

  2. This is not a very good example because

    a. The factors interact (Edward Low is different in his profile). Also, see the warning message.

    b. The model is singular (as warned when the model was fitted), yielding an estimated variance of zero for college)

Russ Lenth
  • 5,922
  • 2
  • 13
  • 21
  • Many thanks, this is very helpful! I know, the model is not the best example, as I am familiarizing with emmeans and lmer. So SE would be the appropriate eff size parameter and it is interpreted as Cohen's d rules of thumb? – atnplab Sep 30 '19 at 09:32
  • No. For Cohen’s effect size, you should not use the SE but rather the total error SD. See the vignette on prediction for an illustration. If you use the SE, then that is the t ratio, not the Cohen d. – Russ Lenth Sep 30 '19 at 13:52
  • 1
    Inspired by this Q, I added a `divisor` argument to some of the contrast functions, so you can do `emmeans(fit, pairwise ~ sex, divisor = 9.246)`. This will be in the next CRAN update, but is available now from the github site rvlenth/emmeans. – Russ Lenth Oct 02 '19 at 19:06
  • 1
    As I found out that @RussLenth added a related function to the package, I will point it out in this comment: https://rdrr.io/cran/emmeans/man/eff_size.html – Nigini Sep 16 '20 at 18:45
-1
library(yarrr)
View(pirates)
library(lme4)
library(lmerTest)

fit <- lmer(weight~ favorite.pirate * sex +(1|college), data = pirates)  
anova(fit, ddf = "Kenward-Roger")

post <- emmeans(fit, pairwise~  sex)  
post
Community
  • 1
  • 1
atnplab
  • 107
  • 1
  • 7