Rewinding input data of TJSONIterator class with TRewindReaderProc

Question

I'm trying to parse JSON in a program built with embarcadero's c++builder (Tokyo 10.2 Update 3), which is not easy considering their severe lack of documentation.

I am using the TJSONIterator Find method that returns true or false if the Path (eg [0]['key'] or car.model['colour']) you give exists in the JSON data, which according to embarcadero's documentation needs a rewinding procedure passed to the constructor of the TJSONIterator class and if it's not there then an exception is thrown stating that.

The rewinding procedure should inherit the _di_TRewindReaderProc interface so here is my class.

class rewindclass : public TJSONIterator::_di_TRewindReaderProc
{
    public:
    void __fastcall Invoke(System::Json::Readers::TJsonReader* AReader)
    {
        //code to rewind Iterator
        Areader->Rewind();
    }
};

I'm not sure what should go into the Invoke function because as I said the documentation is useless. Obviously you have to do something with the TJsonReader that's passed and the only function I can see that could be used is Rewind but I don't think that's it because the only thing the documentation says about the TRewindReaderProc is

Reference to a procedure that rewinds the input data of the specified JSON reader.

Note: TJsonReader.Rewind does not rewind the input data, it resets the state of the JSON 
reader. This procedure must rewind the actual data stream that provides the input data 
of the JSON reader.

and I cannot see what else could be used instead. It says the actual data stream that provides the input must be reset but I'm not sure how to do this.

I'm using a TStringReader to read in the JSON data which is fed into a TJsonTextReader class constructor and that's fed into a TJSONIterator class constructor with a class that is using the _di_TRewindReaderProc interface.

//create rewindclass
rewindclass *rewind = new rewindclass();

//setting up TJSONIterator class
TStringReader *sread = new TStringReader(this->Memo1->Text);
TJsonTextReader *jread = new TJsonTextReader(sread);
TJSONIterator *jit = new TJSONIterator(jread, *rewind);

This code compiles ok but when I debug it and step into the TJSONIterator constructor the TJsonTextReader is not passed through and because of that when I call the Find method a second time it throws an exception saying no callback procedure set.

So does anyone know why the _di_TRewindReaderProc is not being passed through and what should go into the Invoke method?

score 0 · Answer 1 · answered Jun 10 '18 at 21:20

I struggled for hours with the same problem in Delphi and finally I managed to find a way to make it works. I was using a TFileStream and TStreamReader as underlying readers instead of a TStringReader but I hope you understand the concepts behind any reader. This is my code:

procedure LoadFromFile(AFileName: string);
var
  FS: TFileStream;
  StreamReader: TStreamReader;
  JTReader: TJsonTextReader;
  JIterator: TJSONIterator;

  RewindProc: TJSONIterator.TRewindReaderProc;

begin
  RewindProc := procedure (AReader: TJsonReader)
  begin
    TStreamReader(TJsonTextReader(AReader).Reader).DiscardBufferedData;
    TStreamReader(TJsonTextReader(AReader).Reader).BaseStream.Seek(0, TSeekOrigin.soBeginning);
  end;

  FS:=TFileStream.Create(AFileName, fmOpenRead);
  StreamReader:=TStreamReader.Create(FS);
  JTReader:=TJsonTextReader.Create(StreamReader);
  JIterator:=TJSONIterator.Create(JTReader , RewindProc);

  JIterator.Find('some.path.here');
  JIterator.Find('other.path.here');

end;

The lines inside the RewindProc procedure make the magic. As documentation said, you must make the rewind of the underlying readers and that is what I did with these two lines. The first one clean the internal buffer of the StreamReader and the second one move the file pointer to the beginning.

I hope this help you in some way because the documentation is really poor and I can't find any article on Internet about how to set up properly the rewind procedure to be passed to the JSON iterator.

PD: Then I found some other troubles working with TIterator.Find but that is other matter so I focused on your specific question.

msohn · Answer 2 · 2019-05-21T09:33:10.100

Since TJSONIterator alreadys calls Rewind on the reader, before it calls your Rewind procedure, calling Rewind again is not necessary. Instead reset the stream and discard all buffers of the Reader:

procedure TForm1.Button1Click(Sender: TObject);
const
  JsonRec = '{"some":{"path":{"there":"ahi", "here":"acqui"}}}';
var
  StringStream: TStringStream;
  StreamReader: TStreamReader;
  JsonTextReader: TJsonTextReader;
  Iterator: TJSONIterator;
begin
  JsonTextReader:= nil;
  Iterator:= nil;
  StringStream:= TStringStream.Create(JsonRec);
  try
    StreamReader:= TStreamReader.Create(StringStream);
    JsonTextReader:= TJsonTextReader.Create(StreamReader);
    Iterator:= TJSONIterator.Create(JsonTextReader,
      procedure (AReader: TJSONReader)
      var
        v: TValue;
      begin
        StringStream.Seek(0, soBeginning);
        StreamReader.DiscardBufferedData;
        //workaround for RSP-24517
        v:= TRttiContext.Create.GetType(TJsonTextReader).GetField('FChars').GetValue(AReader);
        v.SetArrayElement(0, #0);
      end);
    if Iterator.Find('some.path.here') then
      ecDebug.Lines.Add(Iterator.AsString);
    if Iterator.Find('some.path.there') then
      ecDebug.Lines.Add(Iterator.AsString);
  finally
    Iterator.Free;
    JsonTextReader.Free;
    StreamReader.Free;
    StringStream.Free;
  end;
end;

There doesn't seem to be a way to reset a TStringReader however, that's why I'm using a TStringStream instead.

Update: I've added the necessary call to DiscardBufferedData in the rewind procedure. This only showed up after tests with bigger files.

Update2: With json files larger than 1K, there's a workaround to a bug in TJsonTextReader required, which fails to clear FChars, so it isn't re-reading the json file after a call to .Rewind which causes an exception "Unexpected character encountered while parsing value...". To access the private FChars, I'm using RTTI as described in https://stackoverflow.com/a/36717896/386473. The bug is logged in QP as https://quality.embarcadero.com/browse/RSP-24517.

Rewinding input data of TJSONIterator class with TRewindReaderProc

2 Answers2