221

I am trying to create a .tar.xz compressed archive in one command. What is the specific syntax for that?

I have tried tar cf - file | xz file.tar.xz, but that does not work.

BenMorel
  • 34,448
  • 50
  • 182
  • 322
George K.
  • 2,867
  • 4
  • 19
  • 28

7 Answers7

356

Use the -J compression option for xz. And remember to man tar :)

tar cfJ <archive.tar.xz> <files>

Edit 2015-08-10:

If you're passing the arguments to tar with dashes (ex: tar -cf as opposed to tar cf), then the -f option must come last, since it specifies the filename (thanks to @A-B-B for pointing that out!). In that case, the command looks like:

tar -cJf <archive.tar.xz> <files>
  • 9
    adding `v` to the command switches (`tar -cJvf`) to be verbose shows the files being added to the archive. – Stuart Cardall Jan 01 '16 at 19:52
  • 20
    Given the ordering is important with the dash, it's probably best to assume the ordering is always important, and put the `f` last, even without the dash. – mwfearnley Aug 24 '16 at 14:53
  • 2
    Is it possible to pass xz parameters like -e (--extended) ? Apparently it wont use extended by default. – chmike Feb 08 '17 at 14:53
  • 3
    `tar` on macOS seems to support `-J` but the feature is not given in the man page. Had to look it up here. Though they do mention XZ in the section on `--options`... – cbarrick Feb 08 '18 at 03:34
  • 5
    @chmike Use XZ_OPT environment variable. E.g. XZ_OPT="-9e -T0". – Dzenly May 22 '20 at 06:27
  • run `export XZ_DEFAULTS="-T 0"` before to use all cores – bb1950328 Oct 15 '22 at 22:38
  • To be a little more explicit, I prefer this version of the command: `tar --xz -cf myarchive.tar.xz path/to/my/dir` – Gabriel Staples Apr 22 '23 at 20:09
  • @Dzenly where is this XZ_OPT environment variable documented? I looked in man and info page of `tar` but didn't find it anywhere (even no generic "_OPT") – MacMartin Jun 27 '23 at 13:02
77

Switch -J only works on newer systems. The universal command is:

To make .tar.xz archive

tar cf - directory/ | xz -z - > directory.tar.xz

Explanation

  1. tar cf - directory reads directory/ and starts putting it to TAR format. The output of this operation is generated on the standard output.

  2. | pipes standard output to the input of another program...

  3. ... which happens to be xz -z -. XZ is configured to compress (-z) the archive from standard input (-).

  4. You redirect the output from xz to the tar.xz file.

mpen
  • 272,448
  • 266
  • 850
  • 1,236
Wojciech Adam Koszek
  • 1,050
  • 8
  • 11
  • 17
    `-f` does not mean "from file" when passed to `xz`. Instead it is short for `--force` (see [xz(1)](http://linux.die.net/man/1/xz) for details), and is best not used unless needed. – Eliah Kagan Apr 16 '15 at 17:41
  • @EliahKagan Fixed. Thanks. – Wojciech Adam Koszek Jan 05 '18 at 06:22
  • 6
    To use multithreaded compression the option -T0 can be used: `tar cf - directory/ | xz -z -T0 - > directory.tar.xz` – oidualc Nov 10 '18 at 21:52
  • 2
    Another advantage is that you can easily specify the compression level or any other option to xz on the command line. For example if you want to speed up the compression and do not care that much about the size you can use `-1 -T0` or `-0 -T0` as options, which will usually still give you a smaller file than gzip in a comparable or faster time, while the default `-6` is considerably slower than gzip. – fgwaller Dec 22 '18 at 03:15
45

If you like the pipe mode, this is the most clean solution:

tar c some-dir | xz > some-dir.tar.xz

It's not necessary to put the f option in order to deal with files and then to use - to specify that the file is the standard input. It's also not necessary to specify the -z option for xz, because it's default.

It works with gzip and bzip2 too:

tar c some-dir | gzip > some-dir.tar.gz

or

tar c some-dir | bzip2 > some-dir.tar.bz2

Decompressing is also quite straightforward:

xzcat tarball.tar.xz | tar x
bzcat tarball.tar.bz2 | tar x
zcat tarball.tar.gz | tar x

If you have only tar archive, you can use cat:

cat archive.tar | tar x

If you need to list the files only, use tar t.

Rafael van Horn
  • 591
  • 4
  • 3
15

Quick Solution

tarxz() { tar cf - "$1" | xz -4e > "$1".tar.xz ; }
tarxz name_of_directory

(Notice, not name_of_directory/)

Using xz compression options

If you want to use compression options for xz, or if you are using tar on MacOS, you probably want to avoid the tar -cJf syntax.

According to man xz, the way to do this is:

tar cf - filename | xz -4e > filename.tar.xz

Because I liked Wojciech Adam Koszek's format, but not information:

  1. c creates a new archive for the specified files.
  2. f reads from a directory (best to put this second because -cf != -fc)
  3. - outputs to Standard Output
  4. | pipes output to the next command
  5. xz -4e calls xz with the -4e compression option. (equal to -4 --extreme)
  6. > filename.tar.xz directs the tarred and compressed file to filename.tar.xz

where -4e is, use your own compression options. I often use -k to --keep the original file and -9 for really heavy compression. -z to manually set xz to zip, though it defaults to zipping if not otherwise directed.

To uncompress and untar

To echo Rafael van Horn, to uncompress & untar (see note below):

xz -dc filename.tar.xz | tar x

Note: unlike Rafael's answer, use xz -dc instead of catxz. The docs recommend this in case you are using this for scripting. Best to have a habit of using -d or --decompress instead of unxz as well. However, if you must, using those commands from the command line is fine.

Connor
  • 4,216
  • 2
  • 29
  • 40
  • Any particular reason for not using the `tar -cJf` syntax on MacOS? Just curious, because I was doing that just now and it seemed to be working. – Lo-Tan Sep 14 '18 at 18:20
  • 2
    @Lo-Tan It works if you have tar. When I was using it I didn't have the right version of tar so it didn't work for me. I also wanted to use the compression options for xz, which you can't do if you use tar. Also, the piping syntax is easier for me to remember, so I tend to use that. But to each their own – Connor Sep 14 '18 at 23:05
10

I can never remember which archive switch does what, so these days, I prefer the "auto-compress" feature in newer tar versions (-a or --auto-compress). The command then simply looks like this:

tar caf file.tar.xz file

With that -a option, tar deduces the compression to use automatically from the file ending used for the archive!

codeling
  • 11,056
  • 4
  • 42
  • 71
2

Try this: tar -cf file.tar file-to-compress ; xz -z file.tar

Note:

  1. tar.gz and tar.xz are not the same; xz provides better compression.
  2. Don't use pipe | because this runs commands simultaneously. Using ; or & executes commands one after another.
Randy
  • 14,349
  • 2
  • 36
  • 42
Jhim Preston
  • 73
  • 1
  • 1
  • +1 On RHEL 5.10, I don't see a `J` option, but this works. I think you need RHEL6. – Engineer2021 Mar 24 '14 at 14:50
  • 2
    Why not adjust the commands to pipe the output from tar to xz? – mwfearnley Aug 24 '16 at 14:47
  • @mwfearnley I think the note is merely to emphasize that the commands required to be executed sequentially. Since the file name is used in both command, they won't write or read data from standard output or input. If using the command without specifying the filename, they can be chained up, like `tar -c file-to-compress | xz -z > file.tar.xz`. – Jack Q Oct 18 '17 at 16:54
  • 1
    Using `&` will not launch a command one after another. Instead, it will launch the first one as a background process. – Luis Lavaire. May 18 '20 at 20:16
  • 1
    You probably mean `&&`, which runs commands sequentially. It also stops executing if any of the individual commands returns an error code – Phillip Schmidt Nov 18 '20 at 21:14
2
tar -I 'xz -9 -T0' -cf <archive.tar.xz> <files>

In this approach, you can pass arguments to xz as well. You may change the arguments as you wish. -9 is for compression level and -T0 to use many threads.

Also it works with other compression tools, for example zstd

tar -I 'zstd -18 -T0' -cf <archive.tar.ztd> <files>

This command is only works on gnu-tar. Which comes with the majority of Linux distributions. In Mac, you can install gtar https://formulae.brew.sh/formula/gnu-tar

Oshan Wisumperuma
  • 1,808
  • 1
  • 18
  • 32