TStream to TStringList fails for unicode Delphi

Question

As shown in Test2 of the following code, a TStringList is converted to a TStream and then the TStream is converted back to a TStringList. However, in Delphi 7 Test2 gives the same as Test1. In unicode Delphi, Test2 does not give correct result. Could you help to suggest what is wrong ?

program Project1;

{$APPTYPE CONSOLE}
{$R *.res}

uses
  Classes, SysUtils;

// http://stackoverflow.com/questions/732666
// Converting TMemoryStream to String in Delphi 2009
function MemoryStreamToString(M: TMemoryStream): string;
begin
  SetString(Result, PChar(M.Memory), M.Size div SizeOf(Char));
end;

procedure Test1;
var
  SrcList: TStrings;
  S: String;
  AStream: TStream;
begin
  SrcList := TStringList.Create;
  try
    with SrcList do
    begin
      Add('aa');
      Add('bb');
      Add('cc');
    end;

    S := SrcList.Text;

    AStream := TMemoryStream.Create;
    try
      // AStream.Write(S[1], Length(S));
      // AStream.Write(S[1], Length(S) * SizeOf(Char));
      AStream.Write(Pointer(S)^, Length(S) * SizeOf(Char));

      WriteLn(MemoryStreamToString(TMemoryStream(AStream)));
    finally
      AStream.Free;
    end;
  finally
    SrcList.Free;
  end;
end;

procedure Test2;
var
  SrcList: TStrings;
  S: String;
  AStream: TStream;
  DestList: TStringList;
  I: Integer;
begin
  SrcList := TStringList.Create;
  try
    with SrcList do
    begin
      Add('aa');
      Add('bb');
      Add('cc');
    end;

    S := SrcList.Text;

    AStream := TMemoryStream.Create;
    try
      // AStream.Write(S[1], Length(S));
      // AStream.Write(S[1], Length(S) * SizeOf(Char));
      AStream.Write(Pointer(S)^, Length(S) * SizeOf(Char));

      DestList := TStringList.Create;
      try
        AStream.Position := 0;
        DestList.LoadFromStream(AStream);

        WriteLn(DestList.Text);
      finally
        DestList.Free;
      end;
    finally
      AStream.Free;
    end;
  finally
    SrcList.Free;
  end;
end;

begin
  try
    Test1;
    Test2;
  except
    on E: Exception do
      WriteLn(E.ClassName, ': ', E.Message);
  end;
  Readln;
end.

Aside: replace `DestList := nil; try DestList := TStringList.Create;` with `DestList := TStringList.Create; try` — David Heffernan, Aug 13 '14 at 07:41
@DavidHeffernan Do you mean there will be no exception in the constructor ? When there can be exception in the constructore, it seems that the call to Create should reside in the try-finally. Could you help to comment ? :p — SOUser, Aug 13 '14 at 08:31
If there is an exception raised in the constructor then the destructor will be called, and the assignment to `DestList` will not occur. The correct idiom is as per my comment — David Heffernan, Aug 13 '14 at 08:33
@DavidHeffernan Without `DestList := nil` before `try-finally`, if exception is raised in the constructor and then the destructor is called, `DestList.Free` in the `finally` part will fail. Could you help to comment ? :D — SOUser, Aug 13 '14 at 08:36
Look very closely at the code in my comment. If the constructor raises then `DestList` is indeed never assigned a value. But also, this happens before the `try` and so the `finally` block is not executed. Have a good look around the RTL/VCL source and see which form prevails. — David Heffernan, Aug 13 '14 at 08:39
@DavidHeffernan Could you help to check whether it is correct now ? :D Thank you very much for your time to help ! — SOUser, Aug 13 '14 at 08:51
@DavidHeffernan (Now `Readln` also stays at a correct place! :D) Thank you very much again ! — SOUser, Aug 13 '14 at 08:55

Remy Lebeau · Accepted Answer · 2014-08-12T22:58:08.740

Test1 writes the raw String data as-is into the TMemoryStream and then reads it back as-is into a String, thus everything matches correctly in all versions of Delphi. Test2 works in Delphi 2007 and earlier for a similar reason.

Test2 fails in Delphi 2009 and later because you are not taking into account that TStrings.LoadFrom...() (and TStrings.SaveTo...()) is TEncoding-aware in those versions of Delphi. You are writing UTF-16 encoded data into the TMemoryStream without a BOM in front, and then you are not telling LoadFromStream() that the stream is UTF-16 encoded. It tries to locate a BOM, and when it does not find one, it loads the stream using TEncoding.Default (aka 8bit Ansi) instead (for backwards compatibility with legacy code).

So, in this example, you need to specify the proper encoding of the stream data when loading it in Delphi 2009 and later:

DestList.LoadFromStream(AStream, TEncoding.Unicode);

Thank you for your helpful answer very much ! – SOUser Aug 13 '14 at 07:36 — SOUser, Aug 13 '14 at 07:36

TStream to TStringList fails for unicode Delphi

1 Answers1