5

Is there a way to convert a .bz2 file to a .tar.bz2 file without decompressing the entire thing to disk and then re-compressing? The decompressed size is larger than my drive. Since bz2 operates on blocks, it would seem like you could just decompress a block, pipe it, re-compress it, remove the decompressed block from memory, etc.

I asked this on Ubuntu Forums and didn't find an answer.

Hennes
  • 65,804
  • 7
  • 115
  • 169
endolith
  • 7,704

2 Answers2

3

Update: My original answer doesn't work at all, sorry. tar won't accept a data stream from STDIN as input, so the first command fails.

The only way I can think of to accomplish what you want is to write your own program to add the required tar headers and such around your data stream. Then you could write:

$ bzcat foo.bz2 | stream-to-tar | bzip - > foo.tar.bz2

... and (assuming your program gets the tar format right) you could decompress it with a standard tar xf foo.tar.bz2.

This probably isn't how you want to do it, since it doesn't provide any of the usual advantages of tar'ing the file in the first place.

$ bzcat foo.bz2 | tar cjf foo.tar.bz2 -

Now, the problem is that tar doesn't include any filesystem in it cause all we've given it is a decompressed data stream. That means you need to decompress/untar it like this:

$ tar --to-stdout -xjf foo.tar.bz2 > foo

quack quixote
  • 43,504
0

I think you'll find that the answer is: You don't do this. The compression gained from a .tbz2 file vs. a .bz2 file is pretty minimal if you compressed it with --best. Here is an example over an httpd error log:

 39M ./httpd-error.log
904K ./httpd-error.log.bz2
904K ./httpd-error.log.tbz2

Otherwise, you'll have to do it with a stop by the hard drive.

Jack M.
  • 3,483