0

A DSA key is created with:

openssl genpkey -genparam -algorithm DSA -out dsaparams.pem -pkeyopt dsa_paramgen_bits:1024

openssl genpkey -paramfile dsaparams.pem -out dsakey.pem

When I try to sign, this error is being displayed:

echo 'bacon' > text

openssl pkeyutl -sign -in text -inkey dsakey.pem -out sig
Public Key operation error

It's possible to sign with openssl dgst -sign but not with the commands above.

Why does this happen? The man openssl-pkeyutl says DSA allows signature and verifcation. I'm using OpenSSL 1.1.0g

Kazh
  • 3

1 Answers1

1

TLDR: it seems openssl pkeyutl is designed to sign a hash, but is fed a smaller file where the hash is expected; and some check present in OpenSSL 1.1.0g (but not OpenSSL 1.0.2g) catches that mistake.

FWIW, there was no error when I tried the question's commands with OpenSSL 1.0.2g. But I see 3 reasons that it could fail (and, for the first, should):

  1. openssl pkeyutl is documented to not perform a hash, which is expected to be done externally. So my guess is that newer versions of openssl pkeyutl bark fail nicely when the input file is shorter than the q parameter of the private key, because that's the only sound thing to do. That's at least consistent with the man page:

    In case of RSA, ECDSA and DSA signatures, this utility will not perform hashing on input data but rather use the data directly as input of signature algorithm. Depending on key type, signature type and mode of padding, the maximum acceptable lengths of input data differ. In general, with RSA the signed data can't be longer than the key modulus, in case of ECDSA and DSA the data shouldn't be longer than field size, otherwise it will be silently truncated to field size.

    In other words, if the value of digest is sha1 the input should be 20 bytes long binary encoding of SHA-1 hash function output.

  2. These days, 1024-bit DSA is rightly deprecated, and in some places (including some SSH implementations) support for it has been summarily removed.

  3. These days, SHA-1 is rightly deprecated.

I would suggest at least 2048-bit DSA, SHA-256, and hashing the file before computing the signature, which is best done with openssl dgst which can hash-then-sign, when openssl pkeyutl cannot. I'm not an openssl guru, but I would try investigating the following:

# generate a private key and extract the public key
openssl genpkey -paramfile dsaparams.pem -out dsaprivkey.pem
openssl dsa -in dsaprivkey.pem -pubout > dsapubkey.pem

# create a file "myfile" to be signed
echo 'The Magic Words are Squeamish Ossifrage' > myfile

# create signature "myfile.sig"
openssl dgst -sha256 -sign dsaprivkey.pem myfile > myfile.sig

# verify "myfile" against signature "myfile.sig" and public key
openssl dgst -sha256 -verify dsapubkey.pem -signature myfile.sig myfile

Note: A former attempt made openssl 1.0.2g generate signatures with 160-bit q (perhaps using SHA-1). Per comment, I added -sha256 to openssl dgst, but it made no difference. Experiments suggest it is necessary to use -pkeyopt dsa_paramgen_q_bits:256, even though the man page explicitly states -pkeyopt dsa_paramgen_md:sha256 takes care of that:

dsa_paramgen_md:digest
The digest to use during parameter generation. Must be one of sha1, sha224 or sha256. If set, then the number of bits in q will match the output size of the specified digest and the dsa_paramgen_q_bits parameter will be ignored (..)

fgrieu
  • 856