Result of RSA encryption/decryption has 3 question marks

Question

I am using RSA to encrypt and decrypt small notepad file with 1 or 2 words. After processing file result has 3 question marks on the begging of result.

For example, if i encrypt and then decrypt notepad file with word "Hello" in it, result would be "???Hello". Where did that 3 question marks come from?

This is the code:

    public partial class Form1 : Form
{

    private RSAParameters publicKey;
    private RSAParameters privateKey;

    public string result;

    public Form1()
    {
        InitializeComponent();
        var rsa = new RSACryptoServiceProvider();
        this.publicKey = rsa.ExportParameters(false);
        this.privateKey = rsa.ExportParameters(true);
    }

    private void button1_Click(object sender, EventArgs e)
    {
        openFileDialog1.ShowDialog();
    }

    private void openFileDialog1_FileOk(object sender, CancelEventArgs e)
    {
        textBox1.Text = openFileDialog1.FileName;
    }

    private void button2_Click(object sender, EventArgs e)
    {
        FileStream fileStream = new FileStream(textBox1.Text, FileMode.Open);

        byte[] buffer = new byte[fileStream.Length];
        int len = (int)fileStream.Length;
        fileStream.Read(buffer, 0, len);

        var rsa = new RSACryptoServiceProvider();
        rsa.ImportParameters(publicKey);

        var encrypted = rsa.Encrypt(buffer, false);

        result = Convert.ToBase64String(encrypted);
        MessageBox.Show(result);
    }

    private void button3_Click(object sender, EventArgs e)
    {

        var rsa = new RSACryptoServiceProvider();
        rsa.ImportParameters(privateKey);

        byte[] toDecode = Convert.FromBase64String(result);

        var decrypted = rsa.Decrypt(toDecode, false);

        string msg = Encoding.ASCII.GetString(decrypted);
        MessageBox.Show(msg);
    }
}

Answer 1

It's likely that your input file encoding is UTF8, and you are decoding it into ASCII. Try changing

string msg = Encoding.ASCII.GetString(decrypted);

to

string msg = Encoding.UTF8.GetString(decrypted);

The question marks are generated by the Byte Order Mark (BOM) in front of the text. It is uncommon for UTF-8 which does not require a BOM. It is more common for UTF-16 where endianness is an issue, but as the rest of the plain text seems to decode to ASCII, it can not be UTF-16 encoded.

Note that ASCII cannot show any characters with value 127 (7F in hexadecimals) or higher. The .NET platform seems to silently replace the BOM values with question marks.