Comparing two linear models with anova() in R

Question

I don't quite understand what the p-value in this output means. I don't mean p-values as such, but in this case.

> Model 1: sl ~ le + ky 
> Model 2: sl ~ le   
  Res.Df     RSS Df   Sum of Sq      F Pr(>F) 
1     97 0.51113                              
2     98 0.51211 -1 -0.00097796 0.1856 0.6676

I get something like that, and now I am wondering which model is the better fit. As there is only ONE and not TWO p-values I'm getting confused. I get different pvalues using summary(model1) or summary(model2)

Now if

> fm2<-lm(Y~X+T)

(T being my indicator variable) and

> fm4<-lm(Y~X)

if I do

> anova(fm2,fm4)

this tests the null hypothesis H0: alpha1==alpha2 (Ha: alpha1!=alpha2)c(alpha being my intercept) So it is tested whether it is better to have one intercept (=> alpha1==alpha2), or two intercepts (alpha1!=alpha2)

In this case we would now obviously reject the null Hypotheses, as the p-value is 0.6676.

This would mean we should rather stick with model fm4, as it is more appropriate for our data.

Did I draw the conclusions right? I tried my very best, but I am not sure what the p-value means. As there is only on, this is what I thought it might mean. Can someone clear things up?

Answer 1

Do you mean "would not obviously reject the null hypothesis" (rather than "now obviously reject")? That would seem to make more sense given the rest of your question.

There is only one p-value because there are two models to compare, hence a single comparison (null hypothesis vs alternative, or really in this case null hypothesis vs unspecified alternative). It sounds from what you have said above as though le is a continuous and ky is a categorical predictor, in which case you are comparing a model with a slope and an intercept against (as you said) a model with a single slope and two intercepts. Because the p-value is relatively large, that means that the data do not provide evidence for an additive effect of ky. The simpler model would generally be more appropriate (although be careful with this conclusion, as p-values are constructed to test hypotheses, not to choose among models).

The p-values you get for summary() of each individual model are the p-values for the effects of each of the parameters in each model, conditional on all the other parameters in that model. If your data are perfectly balanced (which is unlikely in a regression design), you should get the same answers from summary and anova, but otherwise the results from anova are generally preferable.

This question is probably more appropriate for http://stats.stackexchange.com , as it is really about statistical interpretation rather than programming ...