How do I iterate through a directory in Common Lisp?

Question

I'm using OpenMCL on Darwin, and I'd like to do something like:

(loop for f in (directory "somedir")
  collect (some-per-file-processing f))

But I can't get directory to return anything other than NIL, and I can't seem to find any good explanation online (other than "its different for each system").

Any pointers?

Answer 1

There are basically two ways to specify pathnames:

• using strings

Strings are obviously depending on the platform: Unix syntax vs. Windows syntax for example.

"/Users/foo/bar.text"  is a valid pathname
"/Users/foo/*/foo.*"   is a valid pathname with two wildcards

You can create a pathname object from a string:

? (pathname "/Users/bar/foo.text")
#P"/Users/bar/foo.text"

The #p above assures that a pathname object (and not a string) is created, when you read it back.

? #P"/Users/bar/foo.text"
#P"/Users/bar/foo.text"

So, internally Common Lisp works with pathname objects, but it lets you use normal strings and makes pathname objects from them if needed.

When Common Lisp sees a pathname that has not all components specified (for example the directory is missing), then it fills in the components from the pathname object that is the value of the variabel *DEFAULT-PATHNAME-DEFAULTS* .

With the function DESCRIBE you can look at the components of a pathname (here Clozure CL):

? (describe (pathname "/Users/bar/*.text"))
#P"/Users/bar/*.text"
Type: PATHNAME
Class: #<BUILT-IN-CLASS PATHNAME>
TYPE: (PATHNAME . #<CCL::CLASS-WRAPPER PATHNAME #x3000401D03BD>)
%PATHNAME-DIRECTORY: (:ABSOLUTE "Users" "bar")
%PATHNAME-NAME: :WILD
%PATHNAME-TYPE: "text"
%PHYSICAL-PATHNAME-VERSION: :NEWEST
%PHYSICAL-PATHNAME-DEVICE: NIL

• using the Lisp functions creating pathname objects

MAKE-PATHNAME is the function and it takes a few keyword arguments to specify the components.

Sometimes it is also useful to create a new pathname based on an existing one:

(make-pathname :name "foo" :defaults (pathname "/Users/bar/baz.text"))

If you use DIRECTORY it is useful to use a pathname with wildcards. DIRECTORY then will return a list of matching pathnames. The name 'DIRECTORY' is slightly misleading, since DIRECTORY does not list the contents of a directory, but lists the matching pathnames for (usually) a pathname with wildcards. The wildcards can match a sequences of characters in components like /foo/s*c/list*.l*". There is also the wild card ** , which is used to match parts of a directory hierachy like /foo/**/test.lisp , which matches all files test.lisp under the directory foo and its subdirectories.

(directory "/Users/foo/Lisp/**/*.lisp")

Above should return a list of all 'lisp' files in '/Users/foo/Lisp/' and all its subdirectories.

To return the .c files in a single directory use:

(directory "/Users/foo/c/src/*.c")

Note that DIRECTORY returns a list of pathname objects (not a list of strings).

? (directory (make-pathname
               :name "md5"
               :type :wild
               :directory '(:absolute "Lisp" "cl-http" "cl-http-342" "server")))
(#P"/Lisp/cl-http/cl-http-342/server/md5.lisp"
 #P"/Lisp/cl-http/cl-http-342/server/md5.xfasl")

Above uses a pathname object that is created by MAKE-PATHNAME. It returns all the files that match /Lisp/cl-http/cl-http-342/server/md5.* .

This is the same as:

(directory "/Lisp/cl-http/cl-http-342/server/md5.*")

which is shorter, but depends on the Unix pathname syntax.