After I sent an S/MIME signed eMail to an insurance company, their exchange server picked up my S/MIME certificate and their reply was S/MIME encrypted. Before, they used my PGP public key to encrypt messages sent to me, which I was able to decrypt. They never signed their own messages.
Despite the certificate and the private S/MIME key are installed in Thunderbird and Apple Mail, none of them are able to decrypt the message. (Thunderbird can't decrypt message. The sender used one of your certificates to encrypt this message. Despite that, Thunderbird couldn't find the certificate and the matching private key.)
Messages I receive from other entities using my S/MIME certificate for encryption can be decrypted by Thunderbird and Apple Mail, including self-sent test messages and messages from swiftmail/PHP.
The private key is not on a smart card but stored in Thunderbird's software security module and Apple's keychain.
Here are some of the mail headers of the unencryptable eMail I received from the insurance company:
Content-Transfer-Encoding: BASE64
Content-Disposition: attachment; filename=smime.p7m
Content-Type: application/pkcs7-mime; name=smime.p7m;
smime-type="enveloped-data"
...
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
...
MIME-Version: 1.0
Decryption by hand as suggested in How to decrypt a broken S/MIME message sent by Outlook? also fails:
$ openssl smime -decrypt -in msg.eml -inkey privatekey.pem -out decrypted.txt
Error decrypting PKCS#7 structure
140735747326920:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22.50.2/libressl/crypto/evp/evp_enc.c:529:
How to find what's wrong on whose end?
Edit:
$ openssl smime -pk7out -in msg.eml | openssl asn1parse
0:d=0 hl=4 l=27205 cons: SEQUENCE
4:d=1 hl=2 l= 9 prim: OBJECT :pkcs7-envelopedData
15:d=1 hl=4 l=27190 cons: cont [ 0 ]
19:d=2 hl=4 l=27186 cons: SEQUENCE
23:d=3 hl=2 l= 1 prim: INTEGER :00
26:d=3 hl=4 l= 457 cons: SET
30:d=4 hl=4 l= 453 cons: SEQUENCE
34:d=5 hl=2 l= 1 prim: INTEGER :00
37:d=5 hl=3 l= 172 cons: SEQUENCE
40:d=6 hl=3 l= 151 cons: SEQUENCE
43:d=7 hl=2 l= 11 cons: SET
45:d=8 hl=2 l= 9 cons: SEQUENCE
47:d=9 hl=2 l= 3 prim: OBJECT :countryName
52:d=9 hl=2 l= 2 prim: PRINTABLESTRING :GB
56:d=7 hl=2 l= 27 cons: SET
58:d=8 hl=2 l= 25 cons: SEQUENCE
60:d=9 hl=2 l= 3 prim: OBJECT :stateOrProvinceName
65:d=9 hl=2 l= 18 prim: PRINTABLESTRING :Greater Manchester
85:d=7 hl=2 l= 16 cons: SET
87:d=8 hl=2 l= 14 cons: SEQUENCE
89:d=9 hl=2 l= 3 prim: OBJECT :localityName
94:d=9 hl=2 l= 7 prim: PRINTABLESTRING :Salford
103:d=7 hl=2 l= 26 cons: SET
105:d=8 hl=2 l= 24 cons: SEQUENCE
107:d=9 hl=2 l= 3 prim: OBJECT :organizationName
112:d=9 hl=2 l= 17 prim: PRINTABLESTRING :COMODO CA Limited
131:d=7 hl=2 l= 61 cons: SET
133:d=8 hl=2 l= 59 cons: SEQUENCE
135:d=9 hl=2 l= 3 prim: OBJECT :commonName
140:d=9 hl=2 l= 52 prim: PRINTABLESTRING :COMODO RSA Client Authentication and Secure Email CA
194:d=6 hl=2 l= 16 prim: INTEGER :7F9285XXXX...
212:d=5 hl=2 l= 13 cons: SEQUENCE
214:d=6 hl=2 l= 9 prim: OBJECT :rsaesOaep
225:d=6 hl=2 l= 0 cons: SEQUENCE
227:d=5 hl=4 l= 256 prim: OCTET STRING [HEX DUMP]:0068E5XXXX...
487:d=3 hl=4 l=26718 cons: SEQUENCE
491:d=4 hl=2 l= 9 prim: OBJECT :pkcs7-data
502:d=4 hl=2 l= 29 cons: SEQUENCE
504:d=5 hl=2 l= 9 prim: OBJECT :aes-256-cbc
515:d=5 hl=2 l= 16 prim: OCTET STRING [HEX DUMP]:CF1A61XXXX...
533:d=4 hl=4 l=26672 prim: cont [ 0 ]