Troubleshooting 'Too many files open' with lsof

Question

I have a Java application running on Linux with PID 25426. When running lsof -p 25426, I noticed:

java    25426 uid  420w  FIFO                0,8      0t0 273664482 pipe
java    25426 uid  421r  FIFO                0,8      0t0 273664483 pipe
java    25426 uid  461r  FIFO                0,8      0t0 273622888 pipe
java    25426 uid  463w  FIFO                0,8      0t0 273633139 pipe
java    25426 uid  464r  FIFO                0,8      0t0 273633140 pipe
java    25426 uid  465r  FIFO                0,8      0t0 273622889 pipe
java    25426 uid  471w  FIFO                0,8      0t0 273623682 pipe
java    25426 uid  472r  FIFO                0,8      0t0 273633141 pipe

How should this result be interpreted?

I am troubleshooting an issue with too many open files and trying to understand whether this observation is relevant.

As application continues to run, number of pipe entries varies (goes up and down).

Answer 1

Definition

• java - The process with the open file.
• 25426 - This should be the real PID. If not please let us know what it is by posting the header.
• 420 w - The file descriptor number followed by the mode it was opened with. (Read / write)
• 0,8 - Major minor device identification.
• 273664482 - The inode of the file.
• pipe - A FIFO pipe that is open in your application.

Interpretation

You are not closing all your streams. There are many open file descriptors in read or write mode that are writing to un-named pipes. The most common scenario for this to happen, is when folks use Runtime.getRuntime.exec() and then proceed to keep the streams associated with the process open. You can use the commons IO utils library to close them or you can close them yourself.

    try
    {
        p = Runtime.getRuntime().exec("something");
    }
    finally
    {
        if (p != null)
        {
            IOUtils.closeQuietly(p.getOutputStream());
            IOUtils.closeQuietly(p.getInputStream());
            IOUtils.closeQuietly(p.getErrorStream());
        }
    }

If that is not the problem, you'll need to dig into your code base and identify where the leaky streams are and plug them.