WSSE - XML SOAP security and key encryption and storing (EncryptedData/EncryptedKey)

Question

I have now spent the last couple of days to find documentation about this..

I need to send a XML via SOAP with the WSSE security header, but don't know how to encrypt and store the encrypted keys

Here is an example

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
    <soap:Header>
        <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">
            <xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="EK-1B758D26C51BFCD86614340101135741">
                <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-oaep-mgf1p"/>
                <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
                    <wsse:SecurityTokenReference>
                        <wsse:KeyIdentifier EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3">MIIDODCCAiCgAwIBAgIGAU0FlCVCMA0GCSqGSIb3DQEBCwUAMFoxCzAJBgNVBAYTAkRLMRUwEwYDVQQKEwxCYW5rIENvbm5lY3QxFTATBgNVBAsTDEJhbmsgQ29ubmVjdDEdMBsGA1UEAxMUQmFuayBDb25uZWN0IElBLXRlc3QwHhcNMTUwNDI5MTQyODI0WhcNMTgwNDI5MTQyODI0WjAcMRowGAYDVQQDExFiYW5rIGNvbm5lY3QtdGVzdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAI23KdtaRKPTFTe/A1PnsF9dpSlTiXurKmio0OCgTP9wClHwync3JsInRwGTooA20P9zWobUnEFbEiAgRVYCxuYoldRE6NLhSC854/YTjMBeevH1TNa38lpavGiI4UwFhg70U9/JuYs21hoFyzVfaWlVfOkAMm1U/n4wHq6FZW461S5PY4A/UI1Mr8WgeIHU9GqMBtFvjynzq3SLenOPgdmKtyJ3V8EOU+DlgwKmDbxMVMtYNDZtoQvOWnuvlJ6ICDcqcW7OUkmwCKodjxxPvrdaPxyZDhT7h4FgRtrAOS8qR6L7x9D4ZIoxOMPudGvr99OSb4KVtaAEt/R7hKxG3OsCAwEAAaNCMEAwHwYDVR0jBBgwFoAU680YSkZnx1IaJAmI49LlTGiia0wwHQYDVR0OBBYEFMaWOY7Vf/iB3WVA96j5kRtbF8prMA0GCSqGSIb3DQEBCwUAA4IBAQAJ+bssSFWE6KsYT7HSDKag4Eot7yNGMY4Don/MilDnOREdu20QUS131DKrSkpBQiCXbyRUQjUoun4yue0EG+rlG3QUIlNNdJ4KZJB+dTYdLUV7XTYJNPimKAmoZ+PFNvT1eGgWcMT+MbTfpk0mw0V8IprYGa8UPchd6vtSVwpbTcPc/F4bgUTlm/V+FG4bQS61gF0koj0DEZjzat7CBHpozRgfRlXgwu26vnhWGc99uKH4GAKN4JpqPi/6Yz+7iQNJUC3yeezgBxFrIXuLpkBZSP4zunf9VxsICnxkFUXOTuYBdcbhPNzqMknD5ijFcFRZITwdv7x3uJGLkM7iUfBp</wsse:KeyIdentifier>
                    </wsse:SecurityTokenReference>
                </ds:KeyInfo>
                <xenc:CipherData>
                    <xenc:CipherValue>af9+FhA91ytLwjeRvTYJsRCkhjHmAQGwqYwMBoNZBn7BZhF/a6EUpM9ByarVhx1SRCpjW5fb8tBVuJO1ZkjfTUZ5EAh/oDLbkmwPdSAAVzmAURHwCq3XQgMZV3lAczlLnPamxjjZBCGqxvAmBo1CvFFPC4AcBedqY92mP8XGyVHpS7JYKOxqXK2vUA1by7371x+Mu0aoS2zJPyPLa1IPwOYgR9qicmWz1RNPiEVA8ZBCN0NRyg7FLJxdUcE81z+1SjButBo2j3qcwkNcecHzZAnweY+LSWp3H5JA3WNzUHUuvFHEaPzT5jd7fUI16xo8NLK8/Rd8Eq/zDD+T3baeVQ==</xenc:CipherValue>
                </xenc:CipherData>
                <xenc:ReferenceList>
                    <xenc:DataReference URI="#ED-1B758D26C51BFCD86614340101135852"/>
                </xenc:ReferenceList>
            </xenc:EncryptedKey>
        </wsse:Security>
        <technicalAddress xmlns="http://example.com/schema/2014" xmlns:ns2="http://www.w3.org/2000/09/xmldsig#"/>
        <activationHeader xmlns="http://example.com/schema/2014" xmlns:ns2="http://www.w3.org/2000/09/xmldsig#">
            <organisationIdentification>
                <mainRegistrationNumber>8079</mainRegistrationNumber>
                <isoCountryCode>DK</isoCountryCode>
            </organisationIdentification>
            <functionIdentification>112233445566778899</functionIdentification>
            <erpInformation/>
            <endToEndMessageId>d28b6a7dad414014a59029ef1a7e84d4</endToEndMessageId>
            <createDateTime>2015-06-11T10:08:33.258+02:00</createDateTime>
        </activationHeader>
    </soap:Header>
    <soap:Body>
        <xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="ED-1B758D26C51BFCD86614340101135852" Type="http://www.w3.org/2001/04/xmlenc#Content">
            <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#aes256-cbc"/>
            <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
                <wsse:SecurityTokenReference xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse11="http://docs.oasis-open.org/wss/oasis-wss-wssecurity-secext-1.1.xsd" wsse11:TokenType="http://docs.oasis-open.org/wss/oasis-wss-soap-message-security-1.1#EncryptedKey">
                    <wsse:Reference URI="#EK-1B758D26C51BFCD86614340101135741"/>
                </wsse:SecurityTokenReference>
            </ds:KeyInfo>
            <xenc:CipherData>
                <xenc:CipherValue>dTSVuEJ90OYguQOsOz2ZtcE2mybwuvVl19pp7/e5yuvNygx3w5v+prpEvbjYLauiIAB3lrVDK2astJeYJGnDbaVJVeU0YqH5ItYVn7Wz36jJM52KB+UNbYo8EdTKYjsZuADzH+tAoA+pwYxGBXMEQctNI+C711HgP2hbpHNYOG7nAMOIrP/0B3FCy+st+9CbYlwAEENreTYunEEA41hciFnWCsIx0el7OeuiA6V51fAmvrF19RPNKwaptvbvmVdKj//RQ/0U1kRny16mDnFfX92bI3HBQm4XJA0nEfSvio7EUAAdhe77GMfu7+JELqXNowPGPLlvrbCFYnQhxGRITHtTIEbtJA6MKtBzHgjtw5pt7oWxKgGUnaJTfOPOSv43RLFGggkT/+gTjnZOagu8hhXp0x5HXJuZzw90aIS3jAfSPDc2ivct4WhWk0wcuQyC2rAh4I7gtiR+LqJJGqvucw4S+NR95FunKHKEW4yasKW1oU31/rRbp4Bmwo6BPsQlxnaSHPtk68IVkYDBslz1A5gOP+M/Iam2WI02y6sE/7aAH1ruN3pZlVuYFc3JDNHOPOvevP110d60lroknGdc9vxcFfj48OCKw/8Ed6tiXtAvk0Qu9Qt4ZyLUoPKIWEqjdLjwVadTDJQFAxRptNgiCos7s0czadUu7FNCRxfndjDxhA7trvys44ufEyK++YzZIgNu3r4dywNI22Nm+JZtLj+rX8ARE6FTPlxGBD0SSdXsfCfY2N1ytBBHQRnPsVaHK1p7KOhwQVbqEupcGyvaRolnymOzDLGFdS06OGYFrYXdgIbuqYtZP8QerXtUl0sWNAvvqHSPCQcpKecpMEecar+FUVwLEA+H1wzOprCMbRR+EgIboeDqQ7GxXqugkuFyvnlLDgxnaWhEhQb/5kAcQmnyUZ57MhDcUJqqQ4Cdmwrcxho1P+YqWY9yn0E86F+hl5976a/gH5KBobB84OWmgcX42eAmqpJf+8c8SuBv+7NctbQOk21aYlFEpkwSme/kG1/edtyoHQH/hF0RB1cT8g+u9S9AK2rs3s2G+Ap0U5oyY8pqJalGdZSBudE0sU4mhOV8trtx0FrN9A7pNkTcGPH25nCtyIz6rzR+DP8Mtgw5385s5ivVlDb+z74Wbh6iu7ZkVAogNTpUYU/1BxDXWJqFMkFmfziNxQ5AQqm1vGlBzXifoQkUFX1riutNphmu0Hs+7KMmMLvtW2cXmQDpkHFKVheeN4w7pBCEZ8KhZ0VTOwRZcdvrNcpYfXM13/QdTHQmCqqwgS/VvlUFz7PDn0/OKo6moUic8W6b1iEvd3kfc7QkunxoOUoJr4RwJ+PqCzN6PxQivAFA2tmDPc8qEa1PAdxTeNFoR/6dNQRojouuJq3C1LrbmGf6lQPvKi3KeKHXyjmDr7Tve+al2tcWJVr+1qEM3/XuthoiZbuTDxYUjZ2nf2fhHrmNcfvrfNxSNHVdQPp2R9Rf3eGxlRJsmRpef66VbYhOpmiH4xmq45EWiyBZmYm+tZtjsP51EDMIvdFbVRSGO/hMqURrDSsJXJeot27Iup2s0P2n/6a9k0c4SVvf/WXNN5x9JNvjU97bQNDQRfonJmo9pRYYHl1tSqNIYBK7KsMH+qr1vmiJuhrXUuL/RtOKvE9KXQ8kGoC9oF5rFn21z40ElxG5XRTASg==</xenc:CipherValue>
            </xenc:CipherData>
        </xenc:EncryptedData>
    </soap:Body>
</soap:Envelope>

First of all I have never worked with SOAP before so chances I do things wrong has pretty good odds :)

Have found something here, but I need more details https://www.w3.org/TR/2002/REC-xmlenc-core-20021210/Overview.html#aes256-cbc

How are the iv and the key stored in CipherValue in the header?

When sending the XML request to the webservice I get this error

23-08-2018 12:50:02   General exception:Padding is invalid and cannot be removed.
23-08-2018 12:50:02   Stack trace:    at System.Security.Cryptography.CapiSymmetricAlgorithm.DepadBlock(Byte[] block, Int32 offset, Int32 count)
   at System.Security.Cryptography.CapiSymmetricAlgorithm.TransformFinalBlock(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount)
   at System.Security.Cryptography.Xml.EncryptedXml.DecryptData(EncryptedData encryptedData, SymmetricAlgorithm symmetricAlgorithm)
   at SomeClassCore.XmlSecurity.Decryptor.DecryptData(Byte[] symmetricKey)
   at SomeClassCore.SecurityServiceImpl.UnwrapRequest(ServiceRequest serviceRequest)
   at BD.BCA.MessageHandler.MessageHandler.ProcessRequest(HttpContext context)

Have searched a bit more.. Maybe the iv must be a part of the stored data. But it's still not working? Same error as above

class Encryption {
    const AES256_CBC = 'AES-256-CBC';

    public function data_encrypt(string $data, string $cipher): Array{
        switch($cipher){
            case self::AES256_CBC:
                $key_length     = 32;
                $block_length   = 16;
                break;
        }

        $iv     = openssl_random_pseudo_bytes(openssl_cipher_iv_length($cipher));
        $key    = openssl_random_pseudo_bytes($key_length);

        $encrypted_data = $iv.openssl_encrypt($data, $cipher, $key, OPENSSL_RAW_DATA, $iv);

        return [
            'data'  => base64_encode($this->pkcs7_padding($encrypted_data, $block_length)),
            'key'   => $key
        ];
    }

    public function key_encrypt(string $key): string{
        $public_cert = openssl_pkey_get_public('contents of public cert');
        openssl_public_encrypt($key, $data, $public_cert, OPENSSL_PKCS1_OAEP_PADDING);
        openssl_free_key($public_cert);

        return base64_encode($data);
    }

    private function pkcs7_padding(string $data, int $block_length): string{
        $pad = $block_length - (strlen($data) % $block_length);

        return $data.str_repeat(chr($pad), $pad);
    }
}

$Enc = new Encryption;
$data_encrypted = $Enc->data_encrypt('The message I want to encrypt', Encryption::AES256_CBC);

//  This base64 encoded string goes to <EncryptedData>
$data_encrypted['data'];

//  This base64 encoded string goes to <EncryptedKey> in the header
$Enc->key_encrypt($data_encrypted['key']);

update

Have been in contact with the maintainer of the webservice and OAEP padding is used with the RSA encryption and PKCS7 padding is used with AES chipher..

As I can see this is also what I do?

Answer 1

*TESTED AND WORKING CODE * I would suggest that you separate the different parts involved. The most likely cause to your problems is the order of execution (i.e. you should do padding before encryption). I am also surprised that there is no signature, but that might not be required in your case. However, I prepared the suggested code for you to test and also added decrypt/decode functions to make testing easier. Good luck.

<?php

class Encryption {

    const AES256_CBC = 'AES-256-CBC';
    const IV_BYTES = 16;

    protected $binary_security_token = null;
    protected $private_key = null;
    protected $public_key = null;

    public function data_encrypt(string $data, string $password): Array {

        $key            = hash('sha256', $password, true);
        $iv             = openssl_random_pseudo_bytes(self::IV_BYTES);

        $padding        = 16 - (strlen($data) % 16);
        $data          .= str_repeat(chr($padding), $padding);

        $encrypted_data = openssl_encrypt($data, self::AES256_CBC, $key, OPENSSL_RAW_DATA, $iv);
        $encoded_data   = base64_encode($iv . $encrypted_data);

        return [
            'data'  => $encoded_data,
            'key'   => $key
        ];
    }

    public function data_decrypt(string $data, string $password): Array {

        $decoded_data   = base64_decode($data);
        $key            = hash('sha256', $password, true);
        $iv             = substr($decoded_data, 0, self::IV_BYTES);
        $encrypted_data = substr($decoded_data, self::IV_BYTES);

        $decrypted_data = openssl_decrypt($encrypted_data, self::AES256_CBC, $key, OPENSSL_RAW_DATA, $iv);
        $padding        = ord($decrypted_data[strlen($decrypted_data) - 1]); 

        return [
            'data' => substr($decrypted_data, 0, -$padding)
        ];
    }

    public function key_encrypt(string $key): ?string {

        $encoded_data = null;

        if ($this->public_key && openssl_public_encrypt($key, $data, $this->public_key, OPENSSL_PKCS1_OAEP_PADDING)) {
            $encoded_data = base64_encode($data);
        }
        // openssl_free_key($this->public_key);

        return $encoded_data;
    }

    public function key_decrypt(string $data): ?string {

        $decrypted_data = null;

        $decoded_data = base64_decode($data, true);
        if ($this->private_key && openssl_private_decrypt($decoded_data, $decrypted, $this->private_key, OPENSSL_PKCS1_OAEP_PADDING)) {
            $decrypted_data = $decrypted;
        }
        // openssl_free_key($decrypted);

        return $decrypted_data;
    }

    public function generate_keys(): void {

        $config = [ "private_key_bits" => 2048, "private_key_type" => OPENSSL_KEYTYPE_RSA ];
        $resource = openssl_pkey_new($config);

        if (openssl_pkey_export($resource, $this->private_key)) {
            echo "private_key:\n" . $this->private_key . "\n";
            $private_key_file = "private_key.pem";
            file_put_contents("private_key.pem" , $this->private_key);
        }

        $this->public_key = openssl_pkey_get_details($resource);
        $this->public_key = $this->public_key["key"];
        $this->binary_security_token = preg_replace("#-.+-|[\r\n]| #", "", $this->public_key);
        echo "public_key:\n" . $this->public_key . "\n";
        file_put_contents("public_key.pem", $this->public_key);
    }

    public function load_keys(): void {

        $private_key_path = realpath(dirname(__FILE__) . "/private_key.pem");
        if (!$private_key_path) {
            $this->generate_keys();
            return;
        }
        $private_key_contents = file_get_contents($private_key_path);
        if (!$private_key_contents) {
            $this->generate_keys();
            return;
        }
        $public_key_path = realpath(dirname(__FILE__) . "/public_key.pem");
        if (!$public_key_path) {
            $this->generate_keys();
            return;
        }
        $public_key_contents = file_get_contents($public_key_path);
        if (!$public_key_contents) {
            $this->generate_keys();
            return;
        }

        // Signature to see that data is not manipulated, could be performed on an encrypted body. The spec says you only make a signature for what you can see.
        // Is it important to "hide data", "detect manipulated data" or both ...
        $this->binary_security_token = preg_replace("#-.+-|[\r\n]| #", "", $public_key_contents); // BinarySecurityToken for securityToken in Security header
        // ValueType: "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"
        // EncodingType: "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary"

        if (openssl_pkey_export($private_key_contents, $this->private_key)) {
            echo "private_key:\n" . $this->private_key . "\n";
        }

        $public_resource = openssl_pkey_get_public($public_key_contents);
        if ($public_resource) {
            $this->public_key = openssl_pkey_get_details($public_resource);
            $this->public_key = $this->public_key["key"];
            echo "public_key:\n" . $this->public_key . "\n";
        }
    }
}

$enc = new Encryption();

$encrypted = $enc->data_encrypt("The message I want to encrypt", "password");

// This base64 encoded string goes to <EncryptedData>
// $encrypted['data']

// Test that data_encrypt / data_decrypt works (from a terminal)
echo "encrypted data:\n" . $encrypted["data"] . "\n";
$decrypted = $enc->data_decrypt($encrypted["data"], "password");
echo "decrypted data:\n" . $decrypted["data"] . "\n";

// This base64 encoded string goes to <EncryptedKey> in the header
// $enc->key_encrypt($encrypted['key']);
if (version_compare(phpversion(), "7.1.0", ">=")) {
    $enc->load_keys();
    $pwd_hash_pre = bin2hex($encrypted["key"]);
    echo "hex key:" . $pwd_hash_pre . "\n";
    $encrypted_key = $enc->key_encrypt($encrypted["key"]);
    echo "\nencrypted and base64encoded key:" . $encrypted_key . "\n";
    $decrypted_key = $enc->key_decrypt($encrypted_key);
    $pwd_hash_post = bin2hex($decrypted_key);
    echo "\ndecrypted and decoded key:" . $pwd_hash_post . "\n";
    $equal_hashes = $pwd_hash_pre === $pwd_hash_post ? 'true' : 'false';
    echo "password hashes equal:" . $equal_hashes . "\n";
}

Answer 2

Your error suggest that the API failed to decrypt the openssl AES-256-CBC data.

I think the reason is because in your class you are routing the encryption through your pkcs7_padding() function. I believe that by default, as long as you don't specify OPENSSL_ZERO_PADDING in your openssl_encrypt() function that the padding is pkcs7. The block size for all AES encryption is 128bits or 16 bytes.

So in essence you are padding your already padded encryption. So basically I just removed your pkcs7_padding() from your class.

I did test your public key encryption. I was able to use a 2048b rsa key from 2048b certificate and generate an encrypted public key using a PEM formatted certificate. Whether or not it's padded correctly I have no idea. But the OPENSSL_PKCS1_OAEP_PADDING is probably correct.

My guess is that the RSA encryption worked if the API made it to the AES portion.

As far as how you are assembling the data into the XML I don't have a clue.
But is seems reasonable that in the <xenc:EncryptedKey> tag in the cipher value would be the RSA encrypted key and for the <xenc:EncryptedData> tag the cipher value would be the AES ecrypted data. You just have to figure out how the API is getting the IV.

Read your API docs for how they are expecting the IV to be delivered. I will keep looking too if this does not work for you.

I will research on that later. But for know try without manually padding your encryption. Hope it helps.

Another thing to consider is that in your case example you don't need to use an IV. In your AES encryption you are generating a new key for every encryption and then encrypting the AES key via the RSA public key obtained by your certificate.

If you were using the same AES key I would see a need to implement an IV but in this case I don't. Regardless... We need to know if the API expects an IV and if it does, how it is expected to be sent?

class Encryption {

    const AES256_CBC = 'AES-256-CBC';

    public function __construct(){



    }

    public function data_encrypt($data, $cipher){

        switch($cipher){
            case self::AES256_CBC:
                $key_length     = 32;
                $block_length   = 16;
                break;
        }

        $iv     = openssl_random_pseudo_bytes(openssl_cipher_iv_length($cipher));
        $key    = openssl_random_pseudo_bytes($key_length);

        $encrypted_data = $iv . openssl_encrypt($data, $cipher, $key, OPENSSL_RAW_DATA, $iv);

        return [
            'data'  => base64_encode($encrypted_data),
            'key'   => $key //Does this need to be encoded? 
        ];
    }

    //***Important***
    //Make sure you certificate is 1.) a x.509 certificate resource, 2.)A file path that leads to a PEM encoded certificate, or 3.) a PEM formatted key.
    public function key_encrypt($text){

        $keyResource = openssl_pkey_get_public(file_get_contents('path/to/myCert.pem')); //This returns a resource or FALSE.

        if(!$keyResource){
          echo 'Something wrong with certificate.';
        }

        openssl_public_encrypt($text, $cipherText, $keyResource, OPENSSL_PKCS1_OAEP_PADDING);
        openssl_free_key($keyResource); 
        return base64_encode($cipherText);

    }


}

$Enc = new Encryption;

$cipherText = $Enc->data_encrypt('The message I want to encrypt', Encryption::AES256_CBC);

//  This base64 encoded string goes to <EncryptedData>
echo 'AES Data: ' . $cipherText['data'] . '<br><br>';
echo 'AES Key: ' . $cipherText['key'] . '<br><br>';

//  This base64 encoded string goes to <EncryptedKey> in the header
$key = $Enc->key_encrypt($cipherText['key']);

echo 'RSA OAEP Padded Key: ' . $key;