Dynamic and Lexical variables in Common Lisp

Question

I am reading the book 'Practical Common Lisp' by Peter Seibel.

In Chapter 6, "Variables" sections "Lexical Variables and Closures" and "Dynamic, a.k.a. Special, Variables". http://www.gigamonkeys.com/book/variables.html

My problem is that the examples in both sections show how (let ...) can shadow global variables and doesn't really tell the difference between the Dynamic and Lexical vars.

I understand how closures work but I don't really get whats so special about let in this example:

(defvar *x* 10)

(defun foo ()
  (format t "Before assignment~18tX: ~d~%" *x*)
  (setf *x* (+ 1 *x*))
  (format t "After assignment~18tX: ~d~%" *x*))


(defun bar ()
  (foo)
  (let ((*x* 20)) (foo))
  (foo))


CL-USER> (foo)
Before assignment X: 10
After assignment  X: 11
NIL


CL-USER> (bar)
Before assignment X: 11
After assignment  X: 12
Before assignment X: 20
After assignment  X: 21
Before assignment X: 12
After assignment  X: 13
NIL

I feel like there is nothing special is going on here. The outer foo in bar increments the global x, and foo surrounded by let in bar increments the shadowed x. What's the big deal? I don't see how is this supposed to explain the difference between lexical and dynamic variables. Yet the book continues like this:

So how does this work? How does LET know that when it binds x it's supposed to create a dynamic binding rather than a normal lexical binding? It knows because the name has been declared special.12 The name of every variable defined with DEFVAR and DEFPARAMETER is automatically declared globally special.

What would happen if let would bind x using "normal lexical binding"? All in all, what are the differences between dynamic and lexical binding and how is this example special regarding dynamic binding?

Answer 1

What's going on?

You say: feel like there is nothing special is going on here. The outer foo in bar increments the global x, and foo surrounded by let in bar increments the shadowed x. What's the big deal?

The special that's going on here is that LET can shadow the value of *x*. With lexical variables that's not possible.

The code declares *x* to be special via the DEFVAR.

In FOO now the value of *x* is looked up dynamic. FOO will take the current dynamic binding of *x* or, if there is none, the symbol value of the symbol *x*. A new dynamic binding can, for example, be introduced with LET.

A lexical variable on the other hand has to be present in the lexical environment somewhere. LET, LAMBDA, DEFUN and others can introduce such lexical variables. See here the lexical variable x introduced in three different ways:

(let ((x 3))
  (* (sin x) (cos x)))

(lambda (x)
  (* (sin x) (cos x)))

(defun baz (x)
  (* (sin x) (cos x)))

If our code were:

(defvar x 0)

(let ((x 3))
  (* (sin x) (cos x)))

(lambda (x)
  (* (sin x) (cos x)))

(defun baz (x)
  (* (sin x) (cos x)))

Then X were special in all three above cases, because of the DEFVAR declaration, which declares X to be special - globally for all levels. Because of this, there is the convention to declare special variables as *X*. Thus only variables with stars around them are special - by convention. That's a useful convention.

In your code you have then:

(defun bar ()
  (foo)
  (let ((*x* 20))
    (foo))
  (foo))

Since *x* has be declared special via the DEFVAR above in your code, the LET construct introduces a new dynamic binding for *x*. FOO is then called. Since inside FOO the *x* uses dynamic binding, it looks up the current one and finds that *x* is dynamically bound to 20.

The value of a special variable is found in the current dynamic binding.

Local SPECIAL declarations

There are also local special declarations:

(defun foo-s ()
  (declare (special *x*))
  (+ *x* 1))

If the variable had been declared special by a DEFVAR or DEFPARAMETER, then the local special declaration can be omitted.

A lexical variable directly references the variable binding:

(defun foo-l (x)
  (+ x 1))

Let's see it in practice:

(let ((f (let ((x 10))
           (lambda ()
             (setq x (+ x 1))))))
  (print (funcall f))    ; form 1
  (let ((x 20))          ; form 2
    (print (funcall f))))

Here all variables are lexical. In form 2 the LET will not shadow the X in our function f. It can't. The function uses the lexical bound variable, introduced by the LET ((X 10). Surrounding the call with another lexically bound X in form 2 has no effect on our function.

Let's try special variables:

(let ((f (let ((x 10))
           (declare (special x))
           (lambda ()
             (setq x (+ x 1))))))
  (print (funcall f))    ; form 1
  (let ((x 20))          ; form 2
    (declare (special x))
    (print (funcall f))))

What now? Does that work?

It does not!

The first form calls the function and it tries to look up the dynamic value of X and there is none. We get an error in form 1: X is unbound, because there is no dynamic binding in effect.

Form 2 would work, since the LET with the special declaration introduces a dynamic binding for X.