I'm running into a problem using the commons compress library to create a tar.gz of a directory. I have a directory structure that is as follows.
parent/
child/
file1.raw
fileN.raw
I'm using the following code to do the compression. It runs fine without exceptions. However, when I try to decompress that tar.gz, I get a single file with the name "childDirToCompress". Its the correct size so the files have clearly been appended to each other in the tarring process. The desired output would be a directory. I can't figure out what I'm doing wrong. Can any wise commons compresser set me upon the correct path?
CreateTarGZ() throws CompressorException, FileNotFoundException, ArchiveException, IOException {
File f = new File("parent");
File f2 = new File("parent/childDirToCompress");
File outFile = new File(f2.getAbsolutePath() + ".tar.gz");
if(!outFile.exists()){
outFile.createNewFile();
}
FileOutputStream fos = new FileOutputStream(outFile);
TarArchiveOutputStream taos = new TarArchiveOutputStream(new GZIPOutputStream(new BufferedOutputStream(fos)));
taos.setBigNumberMode(TarArchiveOutputStream.BIGNUMBER_STAR);
taos.setLongFileMode(TarArchiveOutputStream.LONGFILE_GNU);
addFilesToCompression(taos, f2, ".");
taos.close();
}
private static void addFilesToCompression(TarArchiveOutputStream taos, File file, String dir) throws IOException{
taos.putArchiveEntry(new TarArchiveEntry(file, dir));
if (file.isFile()) {
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(file));
IOUtils.copy(bis, taos);
taos.closeArchiveEntry();
bis.close();
}
else if(file.isDirectory()) {
taos.closeArchiveEntry();
for (File childFile : file.listFiles()) {
addFilesToCompression(taos, childFile, file.getName());
}
}
}
I followed this solution and it worked until I was processing a larger set of files and it randomly crashes after processing 15000 - 16000 files. the following line is leaking file handlers:
IOUtils.copy(new FileInputStream(f), tOut);
and the code crashed with a "Too many open files" error at the OS level The following minor change fix the problem:
FileInputStream in = new FileInputStream(f);
IOUtils.copy(in, tOut);
in.close();