Since you do actually seem to need to export: Your current code doesn't load the private key as exportable, so it can't be exported.  The fix is to assert exportability:
X509Certificate2 x509Certificate =
    new X509Certificate2(privateKeyBytes, "", X509KeyStorageFlags.Exportable);
If that's not enough, then you're encountering the difference between CAPI exportability and CNG exportability (Windows older, and newer, crypto libraries).  If the private key from a PFX/PKCS#12 gets loaded into CNG it's only "encrypted exportable", but ExportParameters is plaintext-export.
There's a workaround, though... export it encrypted, then import that somewhere else with a more flexible export policy, then export again.
This snippet uses the .NET Core 3.0+ ExportPkcs8PrivateKey() method, since that's the format you want your data in, and new .NET 5 PemEncoding class to simplify turning the DER encoded output into PEM+DER output.  If your exporter is on .NET Framework, this is a more complex problem. For .NET Standard 2.0 there's not really a clean solution (reflect call the methods for .NET Core/.NET 5, otherwise use the Windows-specific version for .NET Framework?).
byte[] pkcs8PrivateKey;
using (RSA privateKey = x509Certificate.GetRSAPrivateKey())
{
    pkcs8PrivateKey = ExportPrivateKey(privateKey);
}
File.WriteAllText(
    "tls.cer",
    new string(PemEncoding.Write("CERTIFICATE", x509Certificate.RawData));
File.WriteAllText(
    "tls.key",
    new string(PemEncoding.Write("PRIVATE KEY", pkcs8PrivateKey));
...
private static byte[] ExportPrivateKey(RSA privateKey)
{
    try
    {
        // If it's plaintext exportable, just do the easy thing.
        return privateKey.ExportPkcs8PrivateKey();
    }
    catch (CryptographicException)
    {
    }
    using (RSA exportRewriter = RSA.Create())
    {
        // Only one KDF iteration is being used here since it's immediately being
        // imported again.  Use more if you're actually exporting encrypted keys.
        exportRewriter.ImportEncryptedPkcs8PrivateKey(
            "password",
            privateKey.ExportEncryptedPkcs8PrivateKey(
                "password",
                new PbeParameters(
                    PbeEncryptionAlgorithm.Aes128Cbc,
                    HashAlgorithmName.SHA256,
                    1)),
            out _);
        return exportRewriter.ExportPkcs8PrivateKey();
    }
}