Utilizing multi core for tar+gzip/bzip compression/decompression

gzip tar bzip2 bzip
user1118764 picture user1118764 · Sep 7, 2012 · Viewed 165k times · Source

I normally compress using tar zcvf and decompress using tar zxvf (using gzip due to habit).

I've recently gotten a quad core CPU with hyperthreading, so I have 8 logical cores, and I notice that many of the cores are unused during compression/decompression.

Is there any way I can utilize the unused cores to make it faster?

Answer

Jen picture Jen · Jun 14, 2013

You can also use the tar flag "--use-compress-program=" to tell tar what compression program to use.

For example use: 

tar -c --use-compress-program=pigz -f tar.file dir_to_zip

Related questions

How do I tar a directory of files and folders without including the directory itself?

I typically do: tar -czvf my_directory.tar.gz my_directory What if I just want to include everything (including any hidden system files) in my_directory, but not the directory itself? I don't want: my_directory --- my_file …

Read More
How to uncompress a tar.gz in another directory

I have an archive *.tar.gz How can I uncompress this in a destination directory?

Read More
How to extract filename.tar.gz file

I want to extract an archive named filename.tar.gz. Using tar -xzvf filename.tar.gz doesn't extract the file. it is gives this error: gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error exit delayed from …

Read More