Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: Application Code Slow Following Migration to RAD Studio 2010 Seattle


This question is answered.


Permlink Replies: 9 - Last Post: Apr 4, 2016 9:38 AM Last Post By: Richard Zarr
Richard Zarr

Posts: 74
Registered: 7/1/98
Application Code Slow Following Migration to RAD Studio 2010 Seattle  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 17, 2016 2:14 PM
We have a strange problem. We had an older client application (roughly 130K lines) written with BDS2006. We have upgraded the application to compile on RAD Studio 2010 Seattle by mostly converting 'string' types to 'ansistring' and 'char' to 'ansichar' to maintain the original string types used in BDS2006. We have successfully compiled the new code with some strange behavior.

In DEBUG mode, both versions (compiled with BDS2006 and RAD Studio 2010) transfer data between the host server (TCP connection using sockets) and the client at roughly the same rate. When the RELEASE builds are done, the BDS2006 version downloads in roughly 300 ms where the RAD Studio 2010 version downloads in roughly 3 seconds - 10x worse performance. We have checked our build flags and everything is the same between the two IDEs. Before we start ripping code apart and timing things, I wanted to ask the community if anyone else has seen this symptom and if there was any resolve.

Update: We have traced the slow performance to several routines: Format & GUIDToString (so far) are both slowing the performance way down... they didn't in BDS2006. Any ideas?

Edited by: Richard Zarr on Feb 17, 2016 2:49 PM
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Application Code Slow Following Migration to RAD Studio 2010 Seattle  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 17, 2016 2:58 PM   in response to: Richard Zarr in response to: Richard Zarr
Richard wrote:

We have upgraded the application to compile on RAD Studio 2010 Seattle by
mostly converting 'string' types to 'ansistring' and 'char' to 'ansichar'
to
maintain the original string types used in BDS2006.

You really should not do that. Besides, just changing variable declarations
to Ansi types does not guarantee that the code will actually behave the same
as in the old version, especially if you are passing those Ansi variables
to things that are expecting Unicode variables now, thus causing implicit
comversions. That might be accounting for your "strange behaviors".

In DEBUG mode, both versions (compiled with BDS2006 and RAD Studio
2010) transfer data between the host server (TCP connection using
sockets) and the client at roughly the same rate. When the RELEASE
builds are done, the BDS2006 version downloads in roughly 300 ms where
the RAD Studio 2010 version downloads in roughly 3 seconds - 10x worse
performance.

There is no way for anyone here to diagnose that without seeing what your
code is actually doing.

--
Remy Lebeau (TeamB)
Richard Zarr

Posts: 74
Registered: 7/1/98
Re: Application Code Slow Following Migration to RAD Studio 2010 Seattle  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 18, 2016 6:52 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:
Richard wrote:

We have upgraded the application to compile on RAD Studio 2010 Seattle by
mostly converting 'string' types to 'ansistring' and 'char' to 'ansichar'
to
maintain the original string types used in BDS2006.

You really should not do that. Besides, just changing variable declarations
to Ansi types does not guarantee that the code will actually behave the same
as in the old version, especially if you are passing those Ansi variables
to things that are expecting Unicode variables now, thus causing implicit
comversions. That might be accounting for your "strange behaviors".

In DEBUG mode, both versions (compiled with BDS2006 and RAD Studio
2010) transfer data between the host server (TCP connection using
sockets) and the client at roughly the same rate. When the RELEASE
builds are done, the BDS2006 version downloads in roughly 300 ms where
the RAD Studio 2010 version downloads in roughly 3 seconds - 10x worse
performance.

There is no way for anyone here to diagnose that without seeing what your
code is actually doing.

--
Remy Lebeau (TeamB)

The 'string' vs 'ansistring' in our application had no affect on performance (see below). We were using the string types as a dynamic array in our packet transport (probably should be updated to TBytes, but 'string' (now 'ansistring') would automatically allocate memory). We were selective in the migration and converted only those fields to AnsiString... the rest remain Unicode.

With further investigation we have tracked the performance hit to the 'Format' function in System.SysUtils. If we do this:
function GetTextFromCode(LogEventCode : cardinal) : string;
begin
 Result := format('0x%8.8x (%1.1d)', [LogEventCode, LogEventCode]);
end


we see the 10x performance hit on our logging downloads. Simply removing this statement returns the speed to normal. Examining the GUIDToString function shows it too uses the format command to return the GUID as a string...

Maybe this can shed some additional light on why this is happening...

Update: We also see the same performance hit on the 'DateTimeToString' function. In our formatting routine for our log entries (and probably elsewhere in our object download procedure) we see a noticeable (human eye can catch the time difference easily) speed impact on using these functions... on BDS2006 it had not effect to the UI speed... that was limited by the server's ability to fetch the records and send them. We now see the effects of these formatting routines... comments?

Edited by: Richard Zarr on Feb 18, 2016 7:37 AM

Edited by: Richard Zarr on Feb 18, 2016 7:38 AM
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Application Code Slow Following Migration to RAD Studio 2010Seattle [Edit]
Helpful
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 18, 2016 9:49 AM   in response to: Richard Zarr in response to: Richard Zarr
Richard wrote:

If we do this:

function GetTextFromCode(LogEventCode : cardinal) : string;
begin
Result := format('0x%8.8x (%1.1d)', [LogEventCode, LogEventCode]);
end


we see the 10x performance hit on our logging downloads.

What about this?

function GetTextFromCode(LogEventCode : cardinal) : string;
begin
  Result := '0x' + IntToHex(LogEventCode, 8) + ' (' + IntToStr(LogEventCode) 
+ ')';
end;


Or maybe:

function GetTextFromCode(LogEventCode : cardinal) : string;
var
  s1, s2: string;
begin
  s1 := IntToHex(LogEventCode, 8);
  s2 := IntToStr(LogEventCode);
  Result := '0x' + s1 + ' (' + s2 + ')';
end;


Examining the GUIDToString function shows it too uses the format
command to return the GUID as a string...

You could always use the Win32 StringFromCLSID(), StringFromIID(), or StringfromGUID2()
function instead.

--
Remy Lebeau (TeamB)
Richard Zarr

Posts: 74
Registered: 7/1/98
Re: Application Code Slow Following Migration to RAD Studio 2010Seattle [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 18, 2016 10:12 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:
Richard wrote:

If we do this:

function GetTextFromCode(LogEventCode : cardinal) : string;
begin
Result := format('0x%8.8x (%1.1d)', [LogEventCode, LogEventCode]);
end


we see the 10x performance hit on our logging downloads.

What about this?

function GetTextFromCode(LogEventCode : cardinal) : string;
begin
  Result := '0x' + IntToHex(LogEventCode, 8) + ' (' + IntToStr(LogEventCode) 
+ ')';
end;


Or maybe:

function GetTextFromCode(LogEventCode : cardinal) : string;
var
  s1, s2: string;
begin
  s1 := IntToHex(LogEventCode, 8);
  s2 := IntToStr(LogEventCode);
  Result := '0x' + s1 + ' (' + s2 + ')';
end;


Examining the GUIDToString function shows it too uses the format
command to return the GUID as a string...

You could always use the Win32 StringFromCLSID(), StringFromIID(), or StringfromGUID2()
function instead.

--
Remy Lebeau (TeamB)

Thanks... the IntToHex solution worked without the performance impact. Still wondering about DateTimeToString... we are keeping everything Unicode, but the routine seems to have the same performance hit. Example:

var sTS : string;
 
DateTimeToString( sTS, 'yyyy.mm.dd hh:nn:ss.zzz', TimeStamp);
 


We see the same weird slowing of the log entry parsing. The value of sTS is loaded directly into a TListItem later in the routine...

 LI.SubItems.Add(sTS + ' UTC');


Any ideas why?
David Millington

Posts: 257
Registered: 5/29/05
Re: Application Code Slow Following Migration to RAD Studio 2010 Seattle
Helpful
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 18, 2016 9:53 AM   in response to: Richard Zarr in response to: Richard Zarr
We have upgraded the application to compile on RAD Studio 2010 Seattle by
mostly converting 'string' types to 'ansistring' and 'char' to 'ansichar'
to
maintain the original string types used in BDS2006.

You really should not do that.

When converting, you should not change all string types to AnsiString. It's very much the wrong approach and will cause problems. You should redo this keeping all strings, strings. Convert to Ansi only when you absolutely need to, such as for data transfer to something you haven't been able to upgrade to support Unicode. If you control both sides of the network, then upgrade both apps.

TBytes is the way to go for storing arrays of bytes. Don't use Ansi strings. As far as setting the length automatically goes, it's not hard to call SetLength(), and be aware that bytes with a value of 0, which are valid in binary data, terminate a string. So, don't use strings for things that aren't strings :) If you really want automatic length setting, write a tiny record or class wrapper that does so.

we see the 10x performance hit on our logging downloads. Simply removing this statement returns the speed to normal. Examining the GUIDToString function shows it too uses the format command to return the GUID as a string...

Maybe this can shed some additional light on why this is happening...

Update: We also see the same performance hit on the 'DateTimeToString' function. In our formatting routine for our log entries (and probably elsewhere in our object download procedure) we see a noticeable (human eye can catch the time difference easily) speed impact on using these functions... on BDS2006 it had not effect to the UI speed... that was limited by the server's ability to fetch the records and send them. We now see the effects of these formatting routines... comments?

One guess is that you're suffering the overhead of converting to/from Unicode. (Don't forget this is a lossy conversion, too, so really something you should avoid - another reason to keep "string" as "string".) It does take time to convert from Unicode to any specific code page and while it's not high, depending on how your AnsiStrings are placed and where you've used them, you might be doing a lot of conversions. After all, all the libraries that use strings will be using Unicode strings internally. So you are forcing work to be done each time you use these methods or libraries - work that should be absolutely unnecessary, because your app should be using "string" internally everywhere.

Cheers,

David
Richard Zarr

Posts: 74
Registered: 7/1/98
Re: Application Code Slow Following Migration to RAD Studio 2010 Seattle  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 18, 2016 10:19 AM   in response to: David Millington in response to: David Millington
David Millington wrote:
We have upgraded the application to compile on RAD Studio 2010 Seattle by
mostly converting 'string' types to 'ansistring' and 'char' to 'ansichar'
to
maintain the original string types used in BDS2006.

You really should not do that.

When converting, you should not change all string types to AnsiString. It's very much the wrong approach and will cause problems. You should redo this keeping all strings, strings. Convert to Ansi only when you absolutely need to, such as for data transfer to something you haven't been able to upgrade to support Unicode. If you control both sides of the network, then upgrade both apps.

TBytes is the way to go for storing arrays of bytes. Don't use Ansi strings. As far as setting the length automatically goes, it's not hard to call SetLength(), and be aware that bytes with a value of 0, which are valid in binary data, terminate a string. So, don't use strings for things that aren't strings :) If you really want automatic length setting, write a tiny record or class wrapper that does so.

we see the 10x performance hit on our logging downloads. Simply removing this statement returns the speed to normal. Examining the GUIDToString function shows it too uses the format command to return the GUID as a string...

Maybe this can shed some additional light on why this is happening...

Update: We also see the same performance hit on the 'DateTimeToString' function. In our formatting routine for our log entries (and probably elsewhere in our object download procedure) we see a noticeable (human eye can catch the time difference easily) speed impact on using these functions... on BDS2006 it had not effect to the UI speed... that was limited by the server's ability to fetch the records and send them. We now see the effects of these formatting routines... comments?

One guess is that you're suffering the overhead of converting to/from Unicode. (Don't forget this is a lossy conversion, too, so really something you should avoid - another reason to keep "string" as "string".) It does take time to convert from Unicode to any specific code page and while it's not high, depending on how your AnsiStrings are placed and where you've used them, you might be doing a lot of conversions. After all, all the libraries that use strings will be using Unicode strings internally. So you are forcing work to be done each time you use these methods or libraries - work that should be absolutely unnecessary, because your app should be using "string" internally everywhere.

Cheers,

David

Agreed... it is best to go back and re-write our entire session module for various reasons... one is this packet structure:

 TOurPacket = class
 private
  FPID : cardinal; // Packet ID (used for ACK / NAK of packet)
  FPPT : word; // Packet Payload Type ID
  FFlags : word; // Packet flags (admin, encryption, etc)
  FDGUID : TGUID; // Destination GUID
  FSGUID : TGUID; // Source GUID
  FSID : TSID; // Session ID
  FPayload : ansistring; // Variable size payload (convert to TBytes)
  FCS : word; // Two's complement 16 bit checksum
 public
  constructor Create;
  procedure SaveToStream(Stream : TStream);
  procedure LoadFromStream(Stream : TStream);
 
  function CalcCS : word; // calculates the packet checksum
  // public properties
  property PID : cardinal read FPID write FPID;
  property PPT : word read FPPT write FPPT;
  property Flags : word read FFlags write FFlags;
  property DGUID : TGUID read FDGUID write FDGUID;
  property SGUID : TGUID read FSGUID write FSGUID;
  property SID : TSID read FSID write FSID;
  property Payload : ansistring read FPayload write FPayload; // Convert to TBytes
  property CS : word read FCS write FCS;
 end;


We would need to replace the 'ansistring' with a TBytes type, but add some kind of dynamic memory allocation and clean-up...

The second is the session thread is using deprecated Suspend and Resume synchronization... better to replace it with some other (more modern) synchronization which we use in other threads.

Thanks for the input... yes, good advice!
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Application Code Slow Following Migration to RAD Studio 2010Seattle
Correct
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 18, 2016 10:40 AM   in response to: Richard Zarr in response to: Richard Zarr
Richard wrote:

We would need to replace the 'ansistring' with a TBytes type, but
add some kind of dynamic memory allocation and clean-up...

TBytes is a dynamic array, which is a compiler-managed type. It has the
same dynamic allocation and cleanup that strings have. It it even reference
counted, like a string is. It just doesn't have copy-on-write sematics,
like a string has.

The second is the session thread is using deprecated Suspend and
Resume synchronization...

You should never have been using that in the first place, even before it
was deprecated.

better to replace it with some other (more modern) synchronization
which we use in other threads.

Have the thread wait on a TEvent that you signal when you want to "resume"
it from a "suspended" (waiting) state.

--
Remy Lebeau (TeamB)
Richard Zarr

Posts: 74
Registered: 7/1/98
Re: Application Code Slow Following Migration to RAD Studio 2010 Seattle  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 18, 2016 12:24 PM   in response to: Richard Zarr in response to: Richard Zarr
Still waiting for DateTimeToString resolution on performance hit...

UPDATE:

We are close to a solution that converts our non-Unicode text to Unicode globally across our code base. Once completed, we're going to remove string functions from our transport layer and provide an update on the speed of our old code vs our new solution... stay tuned.

Edited by: Richard Zarr on Mar 4, 2016 11:10 AM

UPDATE

After much investigation, the conclusion is that the slow operation was due to a very large amount of string conversions which previously were not required (pre-unicode vs unicode). We are in the process of completely re-writing our application where strings are used to migrate completely to unicode. We'll consider this question closed and answered.

Edited by: Richard Zarr on Apr 4, 2016 9:34 AM
Richard Zarr

Posts: 74
Registered: 7/1/98
Re: Application Code Slow Following Migration to RAD Studio 2010 Seattle  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Apr 4, 2016 9:38 AM   in response to: Richard Zarr in response to: Richard Zarr
Issue was due to a very large number of string conversions that were not required in the previous versions (Unicode vs. non-Unicode). We are re-writing the application to fix the way we use strings to eliminate the issue.
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02