What is in your Mathematica tool bag?

Question

We all know that Mathematica is great, but it also often lacks critical functionality. What kind of external packages / tools / resources do you use with Mathematica?

I'll edit (and invite anyone else to do so too) this main post to include resources which are focused on general applicability in scientific research and which as many people as possible will find useful. Feel free to contribute anything, even small code snippets (as I did below for a timing routine).

Also, undocumented and useful features in Mathematica 7 and beyond you found yourself, or dug up from some paper/site are most welcome.

Please include a short description or comment on why something is great or what utility it provides. If you link to books on Amazon with affiliate links please mention it, e.g., by putting your name after the link.

---

Packages:

1. LevelScheme is a package that greatly expands Mathematica's capability to produce good looking plots. I use it if not for anything else then for the much, much improved control over frame/axes ticks. Its newest version is called SciDraw, and it will be released sometime this year.
2. David Park's Presentation Package (US$50 - no charge for updates)
3. Jeremy Michelson's grassmannOps package provides resources for doing algebra and calculus with Grassmann variables and operators that have non trivial commutation relations.
4. John Brown's GrassmannAlgebra package and book for working with Grassmann and Clifford algebras.
5. RISC (Research Institute for Symbolic Computation) has a variety of packages for Mathematica (and other languages) available for download. In particular, there is Theorema for automated theorem proving, and the multitude of packages for symbolic summation, difference equations, etc. at the Algorithmic Combinatorics group's software page.

Tools:

1. MASH is Daniel Reeves's excellent Perl script essentially providing scripting support for Mathematica v7. (Now built in as of Mathematica 8 with the -script option.)
2. An alternate Mathematica shell with a GNU readline input (using python, *nix only)
3. ColourMaths package allows you to visually select parts of an expression and manipulate them. http://www.dbaileyconsultancy.co.uk/colour_maths/colour_maths.html

Resources:

1. Wolfram's own repository MathSource has a lot of useful if narrow notebooks for various applications. Also check out the other sections such as

   • Current Documentation,
   • Courseware for lectures,
   • and Demos for, well, demos.

2. The Mathematica Wikibook.

Books:

1. Mathematica programming: an advanced introduction by Leonid Shifrin (web, pdf) is a must read if you want to do anything more than For loops in Mathematica. We have the pleasure of having Leonid himself answering questions here.
2. Quantum Methods with Mathematica by James F. Feagin (amazon)
3. The Mathematica Book by Stephen Wolfram (amazon) (web)
4. Schaum's Outline (amazon)
5. Mathematica in Action by Stan Wagon (amazon) - 600 pages of neat examples and goes up to Mathematica version 7. Visualization techniques are especially good, you can see some of them on the author's Demonstrations Page.
6. Mathematica Programming Fundamentals by Richard Gaylord (pdf) - A good concise introduction to most of what you need to know about Mathematica programming.
7. Mathematica Cookbook by Sal Mangano published by O'Reilly 2010 832 pages. - Written in the well known O'Reilly Cookbook style: Problem - Solution. For intermediates.
8. Differential Equations with Mathematica, 3rd Ed. Elsevier 2004 Amsterdam by Martha L. Abell, James P. Braselton - 893 pages For beginners, learn solving DEs and Mathematica at the same time.

Undocumented (or scarcely documented) features:

7. How to customize Mathematica keyboard shortcuts. See this question.
8. How to inspect patterns and functions used by Mathematica's own functions. See this answer
9. How to achieve consistent size for GraphPlots in Mathematica? See this question.
10. How to produce documents and presentations with Mathematica. See this question.

Answer 1

One of the nice things about the Mathematica notebook interface is that it can evaluate expressions in any language, not just Mathematica. As a simple example, consider creating a new Shell input cell type that passes the contained expression to the operating system shell for evaluation.

First, define a function that delegates evaluation of a textual command to the external shell:

shellEvaluate[cmd_, _] := Import["!"~~cmd, "Text"]

The second argument is needed and ignored for reasons that will become apparent later. Next, we want to create a new style called Shell:

1. Open a new notebook.
2. Select the menu item Format/Edit Stylesheet...
3. In the dialog, beside Enter a style name: type Shell.
4. Select the cell bracket beside the new style.
5. Select the menu item Cell/Show Expression
6. Overwrite the cell expression with the Step 6 Text given below.
7. Once again, select the menu item Cell/Show Expression
8. Close the dialog.

Use the following cell expression as the Step 6 Text:

Cell[StyleData["Shell"],
 CellFrame->{{0, 0}, {0.5, 0.5}},
 CellMargins->{{66, 4}, {0, 8}},
 Evaluatable->True,
 StripStyleOnPaste->True,
 CellEvaluationFunction->shellEvaluate,
 CellFrameLabels->{{None, "Shell"}, {None, None}},
 Hyphenation->False,
 AutoQuoteCharacters->{},
 PasteAutoQuoteCharacters->{},
 LanguageCategory->"Formula",
 ScriptLevel->1,
 MenuSortingValue->1800,
 FontFamily->"Courier"]

Most of this expression was copied directly form the built-in Program style. The key changes are these lines:

 Evaluatable->True,
 CellEvaluationFunction->shellEvaluate,
 CellFrameLabels->{{None, "Shell"}, {None, None}},

Evaluatable enables the SHIFT+ENTER functionality for the cell. Evaluation will call the CellEvaluationFunction passing the cell content and content type as arguments (shellEvaluate ignores the latter argument). CellFrameLabels is just a nicety that let's the user identify that this cell is unusual.

With all of this in place, we can now enter and evaluate a shell expression:

1. In the notebook created in the steps above, create an empty cell and select the cell bracket.
2. Select the menu item Format/Style/Shell.
3. Type a valid operating system shell command into the cell (e.g. 'ls' on Unix or 'dir' on Windows).
4. Press SHIFT+ENTER.

It is best to keep this defined style in a centrally located stylesheet. Furthermore, evaluation functions like shellEvaluate are best defined as stubs using DeclarePackage in init.m. The details of both of these activities are beyond the scope of this response.

With this functionality, one can create notebooks that contain input expressions in any syntax of interest. The evaluation function can be written in pure Mathematica, or delegate any or all parts of the evaluation to an external agency. Be aware that there are other hooks that relate to cell evaluation, like CellEpilog, CellProlog and CellDynamicExpression.

A common pattern involves writing the input expression text to a temporary file, compiling the file in some language, running the program and capturing the output for ultimate display in the output cell. There are plenty of details to address when implementing a full solution of this kind (like capturing error messages properly), but one must appreciate the fact that it is not only possible to do things like this, but practical.

On a personal note, it is features like this that makes the notebook interface the center of my programming universe.

Update

The following helper function is useful for creating such cells:

evaluatableCell[label_String, evaluationFunction_] :=
  ( CellPrint[
      TextCell[
        ""
      , "Program"
      , Evaluatable -> True
      , CellEvaluationFunction -> (evaluationFunction[#]&)
      , CellFrameLabels -> {{None, label}, {None, None}}
      , CellGroupingRules -> "InputGrouping"
      ]
    ]
  ; SelectionMove[EvaluationNotebook[], All, EvaluationCell]
  ; NotebookDelete[]
  ; SelectionMove[EvaluationNotebook[], Next, CellContents]
  )

It is used thus:

shellCell[] := evaluatableCell["shell", Import["!"~~#, "Text"] &]

Now, if shellCell[] is evaluated, the input cell will be deleted and replaced with a new input cell that evaluates its contents as a shell command.