Watch, Follow, &
Connect with Us

Please visit our new home
community.embarcadero.com.


Welcome, Guest
Guest Settings
Help

Thread: Can't retrieve TStreams bigger than around 260.000 bytes from a Server



Permlink Replies: 2 - Last Post: May 16, 2017 5:39 AM Last Post By: Marc Guillot Threads: [ Previous | Next ]
Marc Guillot

Posts: 2
Registered: 5/18/16
Can't retrieve TStreams bigger than around 260.000 bytes from a Server
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 25, 2017 7:14 AM
I have a Delphi 10.1 Berlin Datasnap Server, that can't return Data packets (through a TStream) bigger than around 260.000 bytes.

I have programmed it following the \Object Pascal\DataSnap\FireDAC sample from Delphi, which also gets this problem.

To see it, you can open that sample, set blank the IndexFieldName of the qOrders component on ServerMethodsUnit.pas, and set its SQL property to :

select * from Orders
union
select * from Orders

Now the amount of data to be send is beyond 260.000 bytes, which seems to be the point where you can't retrieve it from the client. Getting a EFDException [FireDAC][Stan]-710. Invalid binary storage format.

The data is sent as a Stream that you get from a FDSchemaAdapter on the server, and you load on another FDSchemaAdpater on the client. The connection between Client and Server is also FireDAC.

This is how the Server returns that Stream :
function TServerMethods.StreamGet: TStream;
begin
  Result := TMemoryStream.Create;
  try
    qCustomers.Close;
    qCustomers.Open;
    qOrders.Close;
    qOrders.Open;
    FDSchemaAdapter.SaveToStream(Result, TFDStorageFormat.sfBinary);
    Result.Position := 0;
  except
    raise;
  end;
end;

And this is how the Client retrieves it :
procedure TClientForm.GetTables;
var
  LStringStream: TStringStream;
begin
  FDStoredProcGet.ExecProc;
  LStringStream := TStringStream.Create(FDStoredProcGet.Params[0].asBlob);
  try
    if LStringStream <> nil then
    begin
      LStringStream.Position := 0;
      DataModuleFDClient.FDSchemaAdapter.LoadFromStream(LStringStream, TFDStorageFormat.sfBinary);
    end;
  finally
    LStringStream.Free;
  end;
end;

The Client doesn't get all the content from the Stream at the Blob parameter. I save the content of the Stream on the Server, and the content that arrives at the Blob parameter on the Client, and they have the same size, but the content of the Blob parameter has its content truncated, and the last few Kbytes are zeroes.

This is how I save on the Server the content that will go to the Stream:

FDSchemaAdapter.SaveToFile('C:\Temp\JSON_Server.json', TFDStorageFormat.sfJSON);

This is how I check what I get on the Client blob parameter:

TFile.WriteAllText('C:\Temp\JSON_Client.json', FDStoredProcGet.Params[0].asBlob);

I can see that the Client gets the data truncated.

Do you know a way to retrieve all the Stream from the Datasnap Server to my Client ?.

Thank you.
celestino mosqu...

Posts: 1
Registered: 7/7/01
Re: Can't retrieve TStreams bigger than around 260.000 bytes from a Server
Click to report abuse...   Click to reply to this thread Reply
  Posted: May 8, 2017 5:47 AM   in response to: Marc Guillot in response to: Marc Guillot
Hi mark

you find any solution to this??

Thnx

Marc Guillot wrote:
I have a Delphi 10.1 Berlin Datasnap Server, that can't return Data packets (through a TStream) bigger than around 260.000 bytes.

I have programmed it following the \Object Pascal\DataSnap\FireDAC sample from Delphi, which also gets this problem.

To see it, you can open that sample, set blank the IndexFieldName of the qOrders component on ServerMethodsUnit.pas, and set its SQL property to :

select * from Orders
union
select * from Orders

Now the amount of data to be send is beyond 260.000 bytes, which seems to be the point where you can't retrieve it from the client. Getting a EFDException [FireDAC][Stan]-710. Invalid binary storage format.

The data is sent as a Stream that you get from a FDSchemaAdapter on the server, and you load on another FDSchemaAdpater on the client. The connection between Client and Server is also FireDAC.

This is how the Server returns that Stream :
function TServerMethods.StreamGet: TStream;
begin
  Result := TMemoryStream.Create;
  try
    qCustomers.Close;
    qCustomers.Open;
    qOrders.Close;
    qOrders.Open;
    FDSchemaAdapter.SaveToStream(Result, TFDStorageFormat.sfBinary);
    Result.Position := 0;
  except
    raise;
  end;
end;

And this is how the Client retrieves it :
procedure TClientForm.GetTables;
var
  LStringStream: TStringStream;
begin
  FDStoredProcGet.ExecProc;
  LStringStream := TStringStream.Create(FDStoredProcGet.Params[0].asBlob);
  try
    if LStringStream <> nil then
    begin
      LStringStream.Position := 0;
      DataModuleFDClient.FDSchemaAdapter.LoadFromStream(LStringStream, TFDStorageFormat.sfBinary);
    end;
  finally
    LStringStream.Free;
  end;
end;

The Client doesn't get all the content from the Stream at the Blob parameter. I save the content of the Stream on the Server, and the content that arrives at the Blob parameter on the Client, and they have the same size, but the content of the Blob parameter has its content truncated, and the last few Kbytes are zeroes.

This is how I save on the Server the content that will go to the Stream:

FDSchemaAdapter.SaveToFile('C:\Temp\JSON_Server.json', TFDStorageFormat.sfJSON);

This is how I check what I get on the Client blob parameter:

TFile.WriteAllText('C:\Temp\JSON_Client.json', FDStoredProcGet.Params[0].asBlob);

I can see that the Client gets the data truncated.

Do you know a way to retrieve all the Stream from the Datasnap Server to my Client ?.

Thank you.
Marc Guillot

Posts: 1
Registered: 10/10/08
Re: Can't retrieve TStreams bigger than around 260.000 bytes from a Server
Click to report abuse...   Click to reply to this thread Reply
  Posted: May 16, 2017 5:39 AM   in response to: celestino mosqu... in response to: celestino mosqu...
Hi Celestino.

No, I didn´t solve it. So I coded a workaround to split the data on smaller packets and re-assemble it on the client.

You can see it here :

http://stackoverflow.com/questions/41854631/cant-retrieve-tstreams-bigger-than-around-260-000-bytes-from-a-datasnap-server

celestino mosquera wrote:
Hi mark

you find any solution to this??

Thnx

Marc Guillot wrote:
I have a Delphi 10.1 Berlin Datasnap Server, that can't return Data packets (through a TStream) bigger than around 260.000 bytes.

I have programmed it following the \Object Pascal\DataSnap\FireDAC sample from Delphi, which also gets this problem.

To see it, you can open that sample, set blank the IndexFieldName of the qOrders component on ServerMethodsUnit.pas, and set its SQL property to :

select * from Orders
union
select * from Orders

Now the amount of data to be send is beyond 260.000 bytes, which seems to be the point where you can't retrieve it from the client. Getting a EFDException [FireDAC][Stan]-710. Invalid binary storage format.

The data is sent as a Stream that you get from a FDSchemaAdapter on the server, and you load on another FDSchemaAdpater on the client. The connection between Client and Server is also FireDAC.

This is how the Server returns that Stream :
function TServerMethods.StreamGet: TStream;
begin
  Result := TMemoryStream.Create;
  try
    qCustomers.Close;
    qCustomers.Open;
    qOrders.Close;
    qOrders.Open;
    FDSchemaAdapter.SaveToStream(Result, TFDStorageFormat.sfBinary);
    Result.Position := 0;
  except
    raise;
  end;
end;

And this is how the Client retrieves it :
procedure TClientForm.GetTables;
var
  LStringStream: TStringStream;
begin
  FDStoredProcGet.ExecProc;
  LStringStream := TStringStream.Create(FDStoredProcGet.Params[0].asBlob);
  try
    if LStringStream <> nil then
    begin
      LStringStream.Position := 0;
      DataModuleFDClient.FDSchemaAdapter.LoadFromStream(LStringStream, TFDStorageFormat.sfBinary);
    end;
  finally
    LStringStream.Free;
  end;
end;

The Client doesn't get all the content from the Stream at the Blob parameter. I save the content of the Stream on the Server, and the content that arrives at the Blob parameter on the Client, and they have the same size, but the content of the Blob parameter has its content truncated, and the last few Kbytes are zeroes.

This is how I save on the Server the content that will go to the Stream:

FDSchemaAdapter.SaveToFile('C:\Temp\JSON_Server.json', TFDStorageFormat.sfJSON);

This is how I check what I get on the Client blob parameter:

TFile.WriteAllText('C:\Temp\JSON_Client.json', FDStoredProcGet.Params[0].asBlob);

I can see that the Client gets the data truncated.

Do you know a way to retrieve all the Stream from the Datasnap Server to my Client ?.

Thank you.
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02