Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: Weird behavior when copying tbytes



Permlink Replies: 8 - Last Post: Nov 17, 2015 1:55 AM Last Post By: Rudy Velthuis (...
David Drouin

Posts: 16
Registered: 11/11/11
Weird behavior when copying tbytes
Click to report abuse...   Click to reply to this thread Reply
  Posted: Oct 27, 2015 12:33 PM
procedure WhyIsItNotWorking;
var
Stream : TBytesStream;
b : TBytes ;
begin
Stream := TBytesStream.Create;
Stream.LoadFromFile( AttachedPDF );

b := Stream.Bytes ;

Stream := TBytesStream.Create( b );
Stream.SaveToFile( AttachedPDF + '2' );

FreeAndNil(Stream);
end;

2nd file is corrupted. there is a bunch of wrong bytes at the end of the file. if you notepad the 2nd pdf and remove said bytes. everything is normal

http://rghost.net/private/67PGPC8QF/cef7cc02159e269f3cccaf86fec0bb41

this is a form/fillable pdf

above code works with 'regular' pdf

Lajos Juhasz

Posts: 801
Registered: 3/14/14
Re: Weird behavior when copying tbytes
Click to report abuse...   Click to reply to this thread Reply
  Posted: Oct 27, 2015 1:12 PM   in response to: David Drouin in response to: David Drouin
David Drouin wrote:

procedure WhyIsItNotWorking;
var
Stream : TBytesStream;
b : TBytes ;
begin
Stream := TBytesStream.Create;
Stream.LoadFromFile( AttachedPDF );

b := Stream.Bytes ;

Stream := TBytesStream.Create( b );
Stream.SaveToFile( AttachedPDF + '2' );

FreeAndNil(Stream);
end;

2nd file is corrupted. there is a bunch of wrong bytes at the end of
the file. if you notepad the 2nd pdf and remove said bytes.
everything is normal


It's a really stupid bug in the Delphi source (at least in XE5). The
problem is that the property Bytes returns the entire buffer and not
only the used part of it. In your case the buffer is 90112 long but the
file is only 84367 byte long.

To make the code work before getting the bytes you resize the buffer:

    Stream.Size:=Stream.Size; // Truncate the buffer 
 
    b := Stream.Bytes ;


After this Capacity will be also as size 84367 and b will contain the
file.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Weird behavior when copying tbytes
Click to report abuse...   Click to reply to this thread Reply
  Posted: Oct 27, 2015 1:34 PM   in response to: Lajos Juhasz in response to: Lajos Juhasz
Lajos wrote:

Stream.Size:=Stream.Size; // Truncate the buffer

That is relying on a private implementation detail of TMemoryStream - its
SetSize() method not only sets the new Size property value, but it also resizes
the Capacity property to match the new Size. I would not rely on that behavior.
It would be better to set the Capacity property instead of the Size property:

Stream.Capacity := Stream.Size;


--
Remy Lebeau (TeamB)
Lajos Juhasz

Posts: 801
Registered: 3/14/14
Re: Weird behavior when copying tbytes
Click to report abuse...   Click to reply to this thread Reply
  Posted: Oct 27, 2015 1:38 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:

It would be better to set the Capacity
property instead of the Size property:

Stream.Capacity := Stream.Size;

Yes that would be nice, however without a cracking class it gives:

[dcc32 Error] Unit2.pas(36): E2362 Cannot access protected symbol
TMemoryStream.Capacity

This should be fixed and that would be the best.

Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Weird behavior when copying tbytes
Click to report abuse...   Click to reply to this thread Reply
  Posted: Oct 27, 2015 1:50 PM   in response to: Lajos Juhasz in response to: Lajos Juhasz
Lajos wrote:

Yes that would be nice, however without a cracking class it gives:

[dcc32 Error] Unit2.pas(36): E2362 Cannot access protected symbol
TMemoryStream.Capacity

Good catch, I missed that.

--
Remy Lebeau (TeamB)
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Weird behavior when copying tbytes
Click to report abuse...   Click to reply to this thread Reply
  Posted: Oct 30, 2015 5:09 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:


Stream.Capacity := Stream.Size;

Why change the capacity at all? Why not simply do:

b := Copy(Stream.Bytes, Stream.Size);

and do not mess with the internals of the stream? That way, you have
your own properly sized copy of the data, and you can manipulate it at
your heart's desire without upsetting the stream.

--
Rudy Velthuis http://www.rvelthuis.de

"I could not possibly fail to disagree with you less."
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Weird behavior when copying tbytes
Click to report abuse...   Click to reply to this thread Reply
  Posted: Oct 30, 2015 12:57 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy wrote:

Why change the capacity at all?

To resize the stream's inner TBytes inline without having to reallocating+copy
it.

Why not simply do:

b := Copy(Stream.Bytes, Stream.Size);

Because that creates a whole separate copy of the byte data in memory, while
leaving the original copy intact. That is fine for small files, not so good
for large files.

--
Remy Lebeau (TeamB)
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Weird behavior when copying tbytes
Click to report abuse...   Click to reply to this thread Reply
  Posted: Nov 17, 2015 1:55 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:

Rudy wrote:

Why change the capacity at all?

To resize the stream's inner TBytes inline without having to
reallocating+copy it.

Why not simply do:

b := Copy(Stream.Bytes, Stream.Size);

Because that creates a whole separate copy of the byte data in
memory, while leaving the original copy intact. That is fine for
small files, not so good for large files.

I'm not sure I would load a large file into memory at all. This looks
like something used for pretty small files only, and then making a copy
is no biggie.

--
Rudy Velthuis http://www.rvelthuis.de

"Life's under no obligation to give us what we expect."
-- Margaret Mitchell
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Weird behavior when copying tbytes
Click to report abuse...   Click to reply to this thread Reply
  Posted: Oct 27, 2015 1:17 PM   in response to: David Drouin in response to: David Drouin
David wrote:

b := Stream.Bytes ;

TBytes is a dynamic array of bytes. Dynamic arrays are reference counted.
You are not actually making a copy of the Bytes, you are simply taking a
reference to the first stream's underlying array. And worse, you are not
taking into account that the array is allocated with extra padding to reduce
the number of reallocations needed during individual write operations. The
stream's Capacity property specifies how many bytes have been physically
allocated for the array, but its Size property specifies the number of bytes
that have actually been written into the array.

Stream := TBytesStream.Create( b );

The TBytesStream constructor takes a reference of the input array and sets
its Memory property to point directly at the array's allocated memory, it
does not make a copy of the array. And it also sets its Size and Capacity
properties to match the entire allocated size of the array, not the number
of bytes actually written into the array.

You also have a memory leak, because you are not freeing the first stream
before reassigning your Stream variable to point at the second stream.

2nd file is corrupted. there is a bunch of wrong bytes at the end
of the file.

That is the extra padding present in the array when the first stream's Capacity
is greater than its Size. You are not taking that into account.

Try this instead:

procedure ThisShouldWork;
var
  Stream : TBytesStream;
  b : TBytes ;
begin
  Stream := TBytesStream.Create;
  try
    Stream.LoadFromFile(AttachedPDF);
    Stream.Capacity := Stream.Size;
    b := Stream.Bytes;
  finally
    FreeAndNil(Stream);
  end;
  Stream := TBytesStream.Create(b);
  try
    Stream.SaveToFile(AttachedPDF + '2');
  finally
    FreeAndNil(Stream);
  end;
end;


Or this:

procedure ThisShouldWork;
var
  Stream : TBytesStream;
  b : TBytes ;
begin
  Stream := TBytesStream.Create;
  try
    Stream.LoadFromFile(AttachedPDF);
    b := Copy(Stream.Bytes, 0, Stream.Size);
  finally
    FreeAndNil(Stream);
  end;
  Stream := TBytesStream.Create(b);
  try
    Stream.SaveToFile(AttachedPDF + '2');
  finally
    FreeAndNil(Stream);
  end;
end;


Or this:

procedure ThisShouldWork;
var
  Stream : TBytesStream;
  b : TBytes ;
  size: Integer;
begin
  Stream := TBytesStream.Create;
  try
    Stream.LoadFromFile(AttachedPDF);
    b := Stream.Bytes;
    size := Stream.Size;
  finally
    FreeAndNil(Stream);
  end;
  Stream := TBytesStream.Create(nil);
  try
    Stream.WriteBuffer(PByte(b)^, size);
    Stream.SaveToFile(AttachedPDF + '2');
  finally
    FreeAndNil(Stream);
  end;
end;


Or this:

procedure ThisShouldWork;
var
  Stream1, Stream2 : TBytesStream;
begin
  Stream1 := TBytesStream.Create;
  try
    Stream1.LoadFromFile(AttachedPDF);
    Stream2 := TBytesStream.Create(nil);
    try
      Stream2.CopyFrom(Stream1, 0);
      Stream2.SaveToFile(AttachedPDF + '2');
    finally
      FreeAndNil(Stream2);
    end;
  finally
    FreeAndNil(Stream1);
  end;
end;


Or this:

type
  TMyBytesStream = class(TBytesStream)
  public
    constructor Create(const ABytes: TBytes; const ASize: Integer); reintroduce;
  end;
 
constructor TMyBytesStream.Create(const ABytes: TBytes; const ASize: Integer);
begin
  inherited Create(ABytes);
  SetSize(ASize);
end;
 
procedure ThisShouldWork;
var
  Stream : TBytesStream;
  b : TBytes ;
  size: Integer;
begin
  Stream := TBytesStream.Create;
  try
    Stream.LoadFromFile(AttachedPDF);
    b := Stream.Bytes;
    size := Stream.Size;
  finally
    FreeAndNil(Stream);
  end;
  Stream := TMyBytesStream.Create(b, size);
  try
    Stream.SaveToFile(AttachedPDF + '2');
  finally
    FreeAndNil(Stream);
  end;
end;


Of course, there is no reason to use streams at all if you just want to make
a copy of a file:

uses
  ..., Windows;
 
procedure ThisShouldWork;
begin
  CopyFile(PChar(AttachedPDF), PChar(AttachedPDF + '2'), False);
end;


--
Remy Lebeau (TeamB)
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02