Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: ANN: Faster WideString process for good old non Unicode Delphi 6-2007



Permlink Replies: 5 - Last Post: Sep 12, 2014 3:42 PM Last Post By: Arnaud BOUCHEZ
Arnaud BOUCHEZ

Posts: 143
Registered: 2/17/02
ANN: Faster WideString process for good old non Unicode Delphi 6-2007
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 12, 2014 1:36 AM
For pre-Unicode versions of Delphi, the unique way of having UTF-16 native type is to use the WideString type.
This type, under Windows, matched the BSTR managed type, as used by OLE and COM components.

In Delphi, WideString implementation calls directly the corresponding Windows API, and do not use the main Delphi heap manager.
Even if since Vista, this API did have a huge speed-up, it is still in practice much slower than the regular string type. Problems is not about UTF-16 encoding, but about the memory allocation, which is shared among processes, using the Windows global heap, and is much slower than our beloved FastMM4.
Newer versions of Delphi (since Delphi 2009) feature a refactored string = UnicodeString type, which relies on FastMM4 and not the Windows API, and is much faster than WideString.

In a recent internal project, we had to use a lot of +WideString+instances, to support UTF-16 encoding in Delphi 7/2007, involving a lot of text.
It sounded to be very slow, so we had to do something!

This is where our new SynFastWideString unit comes in.

Purpose of this unit is to patch the system.pas unit for older versions of Delphi, so that +WideString+memory allocation would use FastMM4 instead of the slow BSTR Windows API.
It will speed up the +WideString+process a lot, especially when a lot of content is allocated, since FastMM4 is much more aggressive than Windows' global heap and the BSTR slow API. It could be more than 50 times faster, especially when releasing the used memory.
The WideString implementation pattern does NOT feature Copy-On-Write, so is still slower than the string UnicodeString type as implemented since Delphi 2009. This is the reason why this unit won't do anything on Unicode versions of the compiler, since the new string type is to be preferred there.

See https://github.com/synopse/mORMot/blob/master/SynFastWideString.pas
and http://blog.synopse.info/post/2014/09/12/Faster-WideString-process-for-good-old-non-Unicode-Delphi-6-2007
Vladimir Ulchenko

Posts: 248
Registered: 1/12/00
Re: ANN: Faster WideString process for good old non Unicode Delphi 6-2007
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 12, 2014 1:57 AM   in response to: Arnaud BOUCHEZ in response to: Arnaud BOUCHEZ
On Fri, 12 Sep 2014 01:36:36 -0700, Arnaud BOUCHEZ <> wrote:

This is where our new SynFastWideString unit comes in.

interesting
any real-world benchmarks against "native" widestrings and/or caching approach implemented by Andreas in RtlVclOptimize?

--
Vladimir Ulchenko aka vavan
Arnaud BOUCHEZ

Posts: 143
Registered: 2/17/02
Re: ANN: Faster WideString process for good old non Unicode Delphi 6-2007
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 12, 2014 2:45 AM   in response to: Vladimir Ulchenko in response to: Vladimir Ulchenko
Vladimir Ulchenko wrote:
On Fri, 12 Sep 2014 01:36:36 -0700, Arnaud BOUCHEZ <> wrote:

This is where our new SynFastWideString unit comes in.

interesting
any real-world benchmarks against "native" widestrings and/or caching approach implemented by Andreas in RtlVclOptimize?

Only internal non disclosable benchmark.
But timing close to UnicodeString. So you got an idea.

AFAIK Andreas caching was not worth it since Vista.
Our unit is much faster.
But not OLE compatible.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: ANN: Faster WideString process for good old non Unicode Delphi 6-2007
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 12, 2014 10:25 AM   in response to: Arnaud BOUCHEZ in response to: Arnaud BOUCHEZ
Arnaud wrote:

Purpose of this unit is to patch the system.pas unit for older
versions of Delphi, so that +WideString+memory allocation would
use FastMM4 instead of the slow BSTR Windows API.

That is very dangerous to do. WideString is specifically designed to be
binary-compatible and API-compatible with COM. The compiler expects that.
User code expects that. If you redirect WideString to use FastMM (or any
other memory manager, for that matter), you break that compatibility, and
risk crashing people's executables.

--
Remy Lebeau (TeamB)
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: ANN: Faster WideString process for good old non Unicode Delphi6-2007
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 12, 2014 10:27 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy wrote:

If you redirect WideString to use FastMM (or any other memory
manager, for that matter), you break that compatibility, and
risk crashing people's executables.

Nevermind, I see you covered that risk in your blog article.

--
Remy Lebeau (TeamB)
Arnaud BOUCHEZ

Posts: 143
Registered: 2/17/02
Re: ANN: Faster WideString process for good old non Unicode Delphi6-2007
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 12, 2014 3:42 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Yes.
We could not decently copy the whole blog article here!
:-)
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02