In R formulas, why do I have to use the I() function on power terms, like y ~ I(x^3)
I'm trying to get my head around the use of the tilde operator, and associated functions. My 1st question is why does I() need to be used to specify arithmetic operators? For example, these 2 plots generate different results (the former having a straight line, and the latter the expected curve)
x <- c(1:100)
y <- seq(0.1,10,0.1)
plot(y~x^3)
plot(y~I(x^3))
further, both of the following plots also generate the expected result
plot(x^3, y)
plot(I(x^3), y)
My second question is, perhaps the examples I've been using are too simple, but I don't understand where ~ should actually be used.
Answer
The issue here is how formulas are interpreted. The infix operators "+", "*", ":" and "^" have entirely different meanings than when used with numeric vectors. In a formula the tilde separates the left hand side from the right hand side. In formulas the ^ operator is for constructing interactions so that x = x^2 = x^3 rather than the perhaps expected mathematical power. (A variable interacting with itself is just the same variable.) If you had typed (x+y)^2 the R interpreter would have produced (for its own good internal use), not a mathematical: x^2 +2xy +y^2 , but rather a symbolic: x + y +x:y where x:y is an interaction term.
?formula
The I() function acts to convert the argument to "as.is", i.e. what you expect. So I(x^2) would return a vector of values raised to the second power.
The ~ should be thought of as saying "is distributed as" or "is dependent on" when seen in regression functions. It implies an error term in model descriptions which will be of whatever form the regression function assumes or is specifically called for in the parameters for family. The mean for the base level will
generally be labelled "(Intercept)". The function context and arguments may also further determine a link function such as log() or logit() from the family value, but it is also possible to have a non-canonical family/link combination.
The "+" symbol in a formula is not really adding two variables but is usually an implicit request to calculate a regression coefficient(s) for that variable in the context of the rest of the variables that are on the RHS of a formula. The regression functions use `model.matrix and that function will recognize the presence of factors or character vectors in the formula and build a matrix that expand the levels of the discrete components of the formula.
In plot()-ting functions it basically reverses the usual ( x, y ) order of arguments that the plot function usually takes. There was a plot.formula method written so that formulas could be used as a more "mathematical" mode of communicating with R. In the graphics::plot.formula, curve, and 'lattice' and 'ggplot' functions, it governs how multiple factors or numeric vectors are displayed and "facetted".
I learned later that ~ is actually an infix (or prefix) primitive function that creates an R 'call' which can be accessed with list extraction operators. So LHS ~ RHS is equivalent to formula(LHS, RHS, env=new.env() ). All of that is hidden from the typical user, but it can be a facility used by more advanced function authors. The environment where the formula object was created will be tsearched first for the symbols in the LHS and RHS expressions.
The overloading of the "+" operator is discussed in the comments below and is also done in the plotting packages: ggplot2 and gridExtra where is it separating functions that deliver object results, so it acting and as a pass-through and layering operator. The aggregation functions that have a formula method use "+" as an "arrangement" and grouping operator.