How to force download of big files without using too much memory?

Question

I'm trying to serve large zip files to users. When there are 2 concurrent connections, the server runs out of memory (RAM). I increased the amount of memory from 300MB to 4GB (Dreamhost VPS) and then it worked fine.

I need to allow a lot more than 2 concurrent connections. The actual 4GB would allow something like 20 concurrent connections (too bad).

Well, the current code I'm using, needs the double of memory then the actual file size. That's too bad. I want something like "streaming" the file to user. So I would allocate not more than the chunk being served to users.

The following code is the one I'm using in CodeIgniter (PHP framework):

ini_set('memory_limit', '300M'); // it was the maximum amount of memory from my server
set_time_limit(0); // to avoid the connection being terminated by the server when serving bad connection downloads
force_download("download.zip", file_get_contents("../downloads/big_file_80M.zip"));exit;

The force_download function is as follows (CodeIgniter default helper function):

function force_download($filename = '', $data = '')
{
    if ($filename == '' OR $data == '')
    {
        return FALSE;
    }

    // Try to determine if the filename includes a file extension.
    // We need it in order to set the MIME type
    if (FALSE === strpos($filename, '.'))
    {
        return FALSE;
    }

    // Grab the file extension
    $x = explode('.', $filename);
    $extension = end($x);

    // Load the mime types
    @include(APPPATH.'config/mimes'.EXT);

    // Set a default mime if we can't find it
    if ( ! isset($mimes[$extension]))
    {
        $mime = 'application/octet-stream';
    }
    else
    {
        $mime = (is_array($mimes[$extension])) ? $mimes[$extension][0] : $mimes[$extension];
    }

    // Generate the server headers
    if (strpos($_SERVER['HTTP_USER_AGENT'], "MSIE") !== FALSE)
    {
        header('Content-Type: "'.$mime.'"');
        header('Content-Disposition: attachment; filename="'.$filename.'"');
        header('Expires: 0');
        header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
        header("Content-Transfer-Encoding: binary");
        header('Pragma: public');
        header("Content-Length: ".strlen($data));
    }
    else
    {
        header('Content-Type: "'.$mime.'"');
        header('Content-Disposition: attachment; filename="'.$filename.'"');
        header("Content-Transfer-Encoding: binary");
        header('Expires: 0');
        header('Pragma: no-cache');
        header("Content-Length: ".strlen($data));
    }

    exit($data);
}

I tried some chunk based codes that I found in Google, but the file always was delivered corrupted. Probably because of bad code.

Could anyone help me?

Answer 1

There are some ideas over in this thread. I don't know if the readfile() method will save memory, but it sounds promising.