How to pad a file with 0xFF using dd?
This command will pad the output file with zeroes until the file size reaches 100 KB:
dd if=inputFile.bin ibs=1k count=100 of=paddedFile.bin conv=sync
However, I want to pad a file with 0xFFs instead of 0x00s.
As far as I know you can't tell dd to pad a file. But you can tell dd to write into a prepadded file.
First create a file of required size filled with 0xFF:
$ dd if=/dev/zero ibs=1 count=102400 | LC_ALL=C tr "\000" "\377" >paddedFile.bin
100+0 records in
200+0 records out
102400 bytes (102 kB) copied, 0,0114595 s, 8,9 MB/s
alternatively you can use head
$ head -c 102400 /dev/zero | LC_ALL=C tr "\000" "\377" >paddedFile.bin
Result:
$ hexdump -C paddedFile.bin
00000000 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
00019000
Then insert the input file at the beginning of the "padded" file:
$ dd if=inputFile.bin of=paddedFile.bin conv=notrunc
0+1 records in
0+1 records out
8 bytes (8 B) copied, 7,4311e-05 s, 108 kB/s
Example input file:
$ hexdump -C inputFile.bin
00000000 66 6f 6f 0a 62 61 72 0a |foo.bar.|
00000008
Result:
$ hexdump -C paddedFile.bin
00000000 66 6f 6f 0a 62 61 72 0a ff ff ff ff ff ff ff ff |foo.bar.........|
00000010 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
00019000
explanation
$ dd if=inputFile.bin of=paddedFile.bin conv=notrunc
^^^^^^^^^^^^
conv=notrunc tells dd to not truncate the output file.
$ dd if=/dev/zero ibs=1 count=102400 | LC_ALL=C tr "\000" "\377" >paddedFile.bin
^^^^^^
tr expects arguments in octal. \377 is octal for 0xFF.
$ dd if=/dev/zero ibs=1 count=102400 | LC_ALL=C tr "\000" "\377" >paddedFile.bin
^^^^^^^^
tr can be locale dependent so to be safe set the locale to C.
$ dd if=/dev/zero ibs=1 count=102400 | LC_ALL=C tr "\000" "\377" >paddedFile.bin
^^^^^
dd might write out less bytes if it cannot read one block full of bytes quick enough. to be sure about size you need to set block size of 1. head does not have this problem.
read here fore more info: https://unix.stackexchange.com/questions/278443/whats-the-posix-way-to-read-an-exact-number-of-bytes-from-a-file
tl;dr: Go to the very last codeblock, just under the bold text that says "This is the full command I would use in my Debian 12".
It's possible to do this with a single pipeline.
Concatenate the input file and the output from a source of 0xFF bytes, then only use dd to stop writing to the output file after 100 KiB. This can be done in two ways:
# flawed, keep reading
{ cat inputFile.bin && </dev/zero LC_ALL=C tr '\000' '\377'; } | dd ibs=1k count=100 of=paddedFile.bin
# flawed, keep reading
</dev/zero LC_ALL=C tr '\000' '\377' | cat inputFile.bin - | dd ibs=1k count=100 of=paddedFile.bin
I find the first syntax better because when read from left to right it properly shows data flow:
inputFile.bin + what tr prints → paddedFile.bin of the desired size.
The second syntax starts from tr, so it's less elegant in this matter.
Both commands are flawed because dd may read less than ibs and still increase the count. The first command will most likely suffer from this, the second command may be lucky and not suffer by chance; still both may suffer. The only POSIX way to fix this is to use ibs=1:
# OK, portable
… | dd ibs=1 count=102400 of=paddedFile.bin
where … denotes the pre-dd fragment of the pipeline from the first or from the second flawed command (both will work fine now, the flaw was from dd).
If your dd supports iflag=fullblock then this will be OK (but not POSIX):
# OK, not portable
… | dd ibs=1k count=100 of=paddedFile.bin iflag=fullblock
Or if your head supports -c (also not POSIX) then you can use head instead of dd:
# OK, not portable
… | head -c 102400 > paddedFile.bin
Your head may even support -c 100K. This is the full command I would use in my Debian 12:
# OK, not portable, quite elegant
{ cat inputFile.bin && </dev/zero LC_ALL=C tr '\000' '\377'; } | head -c 100K > paddedFile.bin
Note that any non-flawed command in this answer produces paddedFile.bin of the size of exactly 100 KiB (unless a fatal error like access denied or so occurs). This is true even if inputFile.bin is larger than 100 KiB.
A possible improvement on lesmana's answer is to operate on the file in-place. This could be a lot faster for large input files and will also keep sparse files sparse. However, in many situations you don't want to modify your input file, and so this method will be unsuitable.
The following example starts with a large, sparse input file and pads it up to a size of 1GB with FF chars. Simply change newsize to your desired value. As you can see, the dd portion takes only a fraction of a second despite this file being very large.
$ ls -ld inputFile.bin
-rw-rw-r-- 1 … 1073741700 … inputFile.bin
$ hexdump inputFile.bin
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
3fffff80 0000 0000
3fffff84
$ newsize=$((1024 * 1024 * 1024))
$ filesize=$(stat -c "%s" inputFile.bin)
$ padcount=$((newsize - filesize))
$ dd if=/dev/zero ibs=1 count="$padcount" | tr "\000" "\377" >> inputFile.bin
124+0 records in
0+1 records out
124 bytes (124 B) copied, 0.000162309 s, 764 kB/s
$ ls -ld inputFile.bin
-rw-rw-r-- 1 … 1073741824 … inputFile.bin
$ hexdump inputFile.bin
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
3fffff80 0000 0000 ffff ffff ffff ffff ffff ffff
3fffff90 ffff ffff ffff ffff ffff ffff ffff ffff
*
40000000
