Comparing models using Anova

Question

Im trying to compare 6 models using the Anova function. mod1:4 are made using the lm function and mod 5 and 6 are models made using the lmer function. I think this is whats causing this error message, any ideas how i can compare these 6 models?

anova(mod1, mod2, mod3, mod4, mod5, mod6)
Error: $ operator not defined for this S4 class

Using the anova function only makes sense if a) you are feeding it with the same kind of models, e.g. only lm models and b) the models should be nested/sequential, i.e. model 2 should include everything that model 1 has + e.g. new predictors, same goes for mod2 vs, mod 3 and so on. — deschen, Feb 24 '22 at 14:43

score 3 · Answer 1 · answered Feb 24 '22 at 14:57

I would recommend comparing models using compare_performance() from performance package instead of anova().

This way you can compare AIC, BIC, R-squared, and so on.

See the material here: https://easystats.github.io/performance/articles/compare.html

After visual comparison, I would recommend you using test_performance() from the same package.

I would also recommend you to rerun your models using the functions they recommend.

Ben Bolker · Accepted Answer · 2022-02-24T23:37:08.873

anova() can work with a mixture lmer and lm models. However, due to the way that R's type system is set up, it only works if the first argument is an lmer model. That is, the anova.merMod() method (which gets called if the first argument is a [g]lmer model) knows how to deal with lm objects, but the anova.lm() method (which gets called if an lm object is first) doesn't know about merMod objects ...

> library(lme4)
Loading required package: Matrix
> fm1 <- lmer(Reaction ~ Days + (1|Subject), sleepstudy, REML = FALSE)
> fm2 <- lm(Reaction ~ Days, sleepstudy)
> fm3 <- lm(Reaction ~ 1, sleepstudy)
> anova(fm1, fm2, fm3)
Data: sleepstudy
Models:
fm3: Reaction ~ 1
fm2: Reaction ~ Days
fm1: Reaction ~ Days + (1 | Subject)
    npar    AIC    BIC  logLik deviance   Chisq Df Pr(>Chisq)    
fm3    2 1965.0 1971.4 -980.52   1961.0                          
fm2    3 1906.3 1915.9 -950.15   1900.3  60.756  1  6.461e-15 ***
fm1    4 1802.1 1814.8 -897.04   1794.1 106.214  1  < 2.2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
> anova(fm3, fm2, fm1)
Error: $ operator not defined for this S4 class

Comparing models using Anova

2 Answers2