Watch, Follow, &
Connect with Us

Please visit our new home
community.embarcadero.com.


Welcome, Guest
Guest Settings
Help

Thread: Plea from Scientific and Engineering users



Permlink Replies: 595 - Last Post: Jul 19, 2014 7:16 PM Last Post By: Quentin Correll
Andy Kozakewycz

Posts: 46
Registered: 11/9/01
Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 24, 2014 12:51 AM
Can you please stop turning Delphi into a gloried database scripting language and give us support for the functions we really need, we really don't care about databases, please provide a version with no database support and added engineering functionality.

We need integrated complex number support (an i on the end of a number makes it complex), matrices, vectors and ASCII string support (not Ansi or unicode) and numeric conversion routines that do not screw up when an application runs in a different country.

Also give us reduced RTTI and smaller exe's, some applications need to run on embedded windows with limited memory and the change to Unicode and RTTI is really making that difficult, an ASCII stringlist would be lovely.

Ta
Steve Thackery

Posts: 151
Registered: 4/29/06
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 24, 2014 1:15 AM   in response to: Andy Kozakewycz in response to: Andy Kozakewycz
Andy Kozakewycz wrote:

We need integrated complex number support (an i on the end of a
number makes it complex), matrices, vectors and ASCII string support
(not Ansi or unicode) and numeric conversion routines that do not
screw up when an application runs in a different country.

For now, I strongly recommend: http://lohninger.com/

It's a superb suite of exceptional quality (in my experience).

--
SteveT
Bob Devine

Posts: 107
Registered: 8/16/01
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 24, 2014 1:23 AM   in response to: Steve Thackery in response to: Steve Thackery
On 24/06/2014 09:15, Steve Thackery wrote:
For now, I strongly recommend:http://lohninger.com/

It's a superb suite of exceptional quality (in my experience).

+1
Andy Kozakewycz

Posts: 46
Registered: 11/9/01
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 24, 2014 2:10 AM   in response to: Steve Thackery in response to: Steve Thackery
Steve Thackery wrote:
Andy Kozakewycz wrote:

We need integrated complex number support (an i on the end of a
number makes it complex), matrices, vectors and ASCII string support
(not Ansi or unicode) and numeric conversion routines that do not
screw up when an application runs in a different country.

For now, I strongly recommend: http://lohninger.com/

It's a superb suite of exceptional quality (in my experience).

--
SteveT

I have my own suite of 2D and 3D and mathematical core components thanks.

Its not third party addons, its core compiler support that is the issue.

I think embarcadero are missing a trick by not catering for a high spending section of industry.
Charles B

Posts: 21
Registered: 12/26/01
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 17, 2014 8:31 AM   in response to: Andy Kozakewycz in response to: Andy Kozakewycz
On 24/06/2014 10:10, Andy Kozakewycz wrote:
Steve Thackery wrote:
Andy Kozakewycz wrote:

We need integrated complex number support (an i on the end of a
number makes it complex), matrices, vectors and ASCII string support
(not Ansi or unicode) and numeric conversion routines that do not
screw up when an application runs in a different country.

For now, I strongly recommend: http://lohninger.com/

It's a superb suite of exceptional quality (in my experience).

--
SteveT

I have my own suite of 2D and 3D and mathematical core components thanks.

Its not third party addons, its core compiler support that is the issue.

I think embarcadero are missing a trick by not catering for a high spending section of industry.

As a member of said community we use the SDL suite too. However Delphi
is a general purpose language.

Have you considered Matlab?

CB
Arnaud BOUCHEZ

Posts: 143
Registered: 2/17/02
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 24, 2014 3:13 AM   in response to: Andy Kozakewycz in response to: Andy Kozakewycz
Andy Kozakewycz wrote:
Can you please stop turning Delphi into a gloried database scripting language and give us support for the functions we really need, we really don't care about databases, please provide a version with no database support and added engineering functionality.

It was one of the main usages of Delphi since the beginning: using RAD to write DB-aware applications via components.
And I suppose it is where Embarcadero gets most of its money.

But I agree with you: Delphi can do much more than this.
For instance, it could be used to write high-performance n-Tier / SOA solutions.
Or dedicated applications using high-performance computation, just as you need, as far as I can tell.

Andy Kozakewycz wrote:
We need integrated complex number support (an i on the end of a number makes it complex), matrices, vectors and ASCII string support (not Ansi or unicode) and numeric conversion routines that do not screw up when an application runs in a different country.

You have to write all this by hand.
We had to by-pass the RTL for almost all its low-level process for our framework to scale on Win32 a

Andy Kozakewycz wrote:
Also give us reduced RTTI and smaller exe's, some applications need to run on embedded windows with limited memory and the change to Unicode and RTTI is really making that difficult, an ASCII stringlist would be lovely.

Are you describing FreePascal?
Vladimir Ulchenko

Posts: 248
Registered: 1/12/00
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 24, 2014 3:49 AM   in response to: Andy Kozakewycz in response to: Andy Kozakewycz
On Tue, 24 Jun 2014 00:51:31 -0700, Andy Kozakewycz <> wrote:

Can you please stop turning Delphi into a gloried database scripting language and give us support for the functions we really need, we really don't care about databases, please provide a version with no database support and added engineering functionality.

We need integrated complex number support (an i on the end of a number makes it complex), matrices, vectors and ASCII string support (not Ansi or unicode) and numeric conversion routines that do not screw up when an application runs in a different country.

Also give us reduced RTTI and smaller exe's, some applications need to run on embedded windows with limited memory and the change to Unicode and RTTI is really making that difficult, an ASCII stringlist would be lovely.

some people here also have a strange ideas that nobody needs string types other than unicode and cares about the size of their applications
and runtime memory consumption and performance these days :)

--
Vladimir Ulchenko aka vavan
Luigi Sandon

Posts: 353
Registered: 10/15/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 24, 2014 4:34 AM   in response to: Vladimir Ulchenko in response to: Vladimir Ulchenko
some people here also have a strange ideas that nobody needs string types other than unicode and cares about the size of their applications
and runtime memory consumption and performance these days :)

It looks surveys are to blame... <G>
Arthur Hoornweg

Posts: 414
Registered: 6/2/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 26, 2014 6:07 AM   in response to: Luigi Sandon in response to: Luigi Sandon
Luigi Sandon wrote:

It looks surveys are to blame... <G>

You may laugh, but I think they are. I wish the survey had been about "what do you think about the quality of our compiler and linker?" but instead it was -again- about "how do you like the bells and whistles? What more 3rd party stuff should we add?"
Nick Hodges

Posts: 2,414
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 26, 2014 6:47 AM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

what do you think about the quality of our compiler and linker?"

They asked directly about that very thing.

Plus, there is plenty of opportunity for textual feedback.

--
Nick
Delphi Programming is Fun
Arthur Hoornweg

Posts: 414
Registered: 6/2/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 26, 2014 7:52 AM   in response to: Nick Hodges in response to: Nick Hodges
Nick Hodges wrote:

They asked directly about that very thing.

No they didn't. I just double-checked. Plenty of questions regarding specific features, platforms and components but deafening silence about compiler and linker. The features I desire most in Delphi are an optimizing compiler and a truly smart linker that strips away whatever isn't called. Unfortunately Delphi is moving further and further away from that.

Edit: I gave plenty of textual feedback, be sure of that!

Edited by: Arthur Hoornweg on Jun 26, 2014 7:52 AM

Nick Hodges

Posts: 2,414
Registered: 9/22/99
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 26, 2014 2:09 PM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

No they didn't.

Well, then I stand corrected. I was sure they had asked about that.

--
Nick
Delphi Programming is Fun
Bruce McGee

Posts: 1,716
Registered: 9/30/99
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 26, 2014 2:31 PM   in response to: Nick Hodges in response to: Nick Hodges
Nick Hodges wrote:

Arthur Hoornweg wrote:

No they didn't.

Well, then I stand corrected. I was sure they had asked about that.

Lots of questions about targets and platforms, which I suppose are
compiler related, but nothing specifically about the compiler and
linker.

Arthur:
What kind of questions would you have liked to see?

--
Regards,
Bruce McGee
Glooscap Software
Arthur Hoornweg

Posts: 414
Registered: 6/2/98
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 27, 2014 6:17 AM   in response to: Bruce McGee in response to: Bruce McGee
Bruce McGee wrote:

Arthur:
What kind of questions would you have liked to see?

One: Whether the user is satisfied with the quality of the compiled code (size-, stability- and speedwise) and if he or she has a need for specific optimizations. Current versions of Delphi link everything and the kitchen sink into the executable, including a massive lead ball on an iron chain called RTTI whether you use that or not. This overhead is very much an obstacle if you do multi-module or plugin-based development. Using runtime packages is not an option for most of my projects (and the fact that Delphi gets updated twice yearly makes using runtime packages an unattractive option anyhow). So I would very much like the option to make my executables small again, by stripping the RTTI completely away.

Another good question would be if the user is satisfied with the Delphi installer. I have 4 versions of Delphi installed on my SSD drive (inside a VM) and my "ProgramData" folder contains 6.7 GB, most of which is Delphi-related installation stuff that is never used! Why does this stuff need to be on my drive at all? Why not just keep it "in the cloud" ?
Bruce McGee

Posts: 1,716
Registered: 9/30/99
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 27, 2014 7:36 AM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

Bruce McGee wrote:

Arthur:
What kind of questions would you have liked to see?

One: Whether the user is satisfied with the quality of the compiled
code (size-, stability- and speedwise) and if he or she has a need
for specific optimizations. Current versions of Delphi link
everything and the kitchen sink into the executable, including a
massive lead ball on an iron chain called RTTI whether you use that
or not. This overhead is very much an obstacle if you do multi-module
or plugin-based development. Using runtime packages is not an option
for most of my projects (and the fact that Delphi gets updated twice
yearly makes using runtime packages an unattractive option anyhow).
So I would very much like the option to make my executables small
again, by stripping the RTTI completely away.

I would love to see questions like this in a survey.

Failing that, it might take some high profile benchmarks to bring some
of these to the top of the list. There have been a couple of good ones
lately, but not enough.

I'm usually not too concerned about executable size in general, but
this is getting ridiculous. And I'm still deploying applications over
dial-up. Seriously.

And I share your opinion of runtime packages.

Another good question would be if the user is satisfied with the
Delphi installer. I have 4 versions of Delphi installed on my SSD
drive (inside a VM) and my "ProgramData" folder contains 6.7 GB, most
of which is Delphi-related installation stuff that is never used!
Why does this stuff need to be on my drive at all? Why not just keep
it "in the cloud" ?

I have XE through XE6 installed in a single VM. I feel your pain.

--
Regards,
Bruce McGee
Glooscap Software
Mason Wheeler

Posts: 37
Registered: 4/15/05
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 27, 2014 9:28 AM   in response to: Bruce McGee in response to: Bruce McGee
Bruce McGee wrote:
I'm usually not too concerned about executable size in general, but
this is getting ridiculous. And I'm still deploying applications over
dial-up. Seriously.

That's mostly generics, not RTTI. I've been pointing out for years now
that Delphi need proper collapsing of identical generics in the executable
output, but any time I bring it up to anyone who actually works on the
product, Allen Bauer comes up with a bunch of technical objections that
sound like total mumbo-jumbo even to me, and I know compilers pretty
well. Apparently there's something seriously broken in the compiler or
linker architecture, because this should not be difficult.

And I share your opinion of runtime packages.

Packages are awesome and amazing... and a little bit ridiculous with the
way they're incompatible across every single new version. I mean, for
heaven's sake, if Java and .NET have had the secret to cross-version
compatibility figured out for well over a decade, it really can't be that difficult!
Bruce McGee

Posts: 1,716
Registered: 9/30/99
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 27, 2014 11:23 AM   in response to: Mason Wheeler in response to: Mason Wheeler
Mason Wheeler wrote:

And I share your opinion of runtime packages.

Packages are awesome and amazing...

I think of them like deploying DLLs. Only if absolutely necessary.

and a little bit ridiculous with
the way they're incompatible across every single new version.

No argument there.

--
Regards,
Bruce McGee
Glooscap Software
Konstantine Pou...

Posts: 128
Registered: 11/3/06
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 27, 2014 11:53 AM   in response to: Bruce McGee in response to: Bruce McGee
I think of them like deploying DLLs. Only if absolutely necessary.

Same here. Never ever for end customer.
Mason Wheeler

Posts: 37
Registered: 4/15/05
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 27, 2014 12:15 PM   in response to: Bruce McGee in response to: Bruce McGee
Bruce McGee wrote:
Mason Wheeler wrote:

And I share your opinion of runtime packages.

Packages are awesome and amazing...

I think of them like deploying DLLs. Only if absolutely necessary.

I don't. I think of them the way the IDE does: as an exceptionally
powerful object-oriented plugin system that provides nearly limitless
potential for extending your program. And if only they would fix those
ridiculous version compatibility issues, it would be even better.
Konstantine Pou...

Posts: 128
Registered: 11/3/06
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 27, 2014 12:43 PM   in response to: Mason Wheeler in response to: Mason Wheeler
To each their own. I consider those as yet another
customer support issue
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 27, 2014 12:45 PM   in response to: Konstantine Pou... in response to: Konstantine Pou...
Konstantine Poukhov wrote:
To each their own. I consider those as yet another
customer support issue

+1

Dalija Prasnikar
Konstantine Pou...

Posts: 128
Registered: 11/3/06
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 27, 2014 11:51 AM   in response to: Nick Hodges in response to: Nick Hodges
They asked directly about that very thing.

Nick. In my view you've become famous for
telling things without ever bothering to check
only to "stand corrected" later. My first experience
about that habit of yours started with Delphi Turbo
edition.

Taking into account that you're considered sort
of "authority" here I would definitely start
checking statements before posting it.
Luigi Sandon

Posts: 353
Registered: 10/15/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 27, 2014 12:44 AM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
You may laugh, but I think they are.

I wasn't laughing. In another thread it was said survey largely shape the product - thereby, if true - the reason Delphi is in its actual lame state is also up to surveys... and it is also true often reading surveys you understand a given path has been already taken, and they need some data to justify their decisions.
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 24, 2014 5:17 AM   in response to: Vladimir Ulchenko in response to: Vladimir Ulchenko
Vladimir Ulchenko wrote:
On Tue, 24 Jun 2014 00:51:31 -0700, Andy Kozakewycz <> wrote:

Can you please stop turning Delphi into a gloried database scripting language and give us support for the functions we really need, we really don't care about databases, please provide a version with no database support and added engineering functionality.

We need integrated complex number support (an i on the end of a number makes it complex), matrices, vectors and ASCII string support (not Ansi or unicode) and numeric conversion routines that do not screw up when an application runs in a different country.

Also give us reduced RTTI and smaller exe's, some applications need to run on embedded windows with limited memory and the change to Unicode and RTTI is really making that difficult, an ASCII stringlist would be lovely.

some people here also have a strange ideas that nobody needs string types other than unicode and cares about the size of their applications
and runtime memory consumption and performance these days :)

Yep. Some of us do care, but there is nobody on the other side to read the message.

Dalija Prasnikar
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 24, 2014 11:28 PM   in response to: Vladimir Ulchenko in response to: Vladimir Ulchenko
Vladimir Ulchenko wrote:

Also give us reduced RTTI and smaller exe's, some applications need
to run on embedded windows with limited memory and the change to
Unicode and RTTI is really making that difficult, an ASCII
stringlist would be lovely.

some people here also have a strange ideas that nobody needs string
types other than unicode

Nobody really NEEDS them. If you want text, one string type is fine. If
you want to store other data, don't use strings.

But if you really think you need an ASCII-only stringlist, write your
own. It is not rocket science and not too much work either. If you
don't need RTTI, switch it off.

OK, it may be a bummer you don't have syntax for complex types or, say,
for decimals, like some other languages, but that does not mean you
can't use them.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"It may be necessary temporarily to accept a lesser evil, but
one must never label a necessary evil as good."
-- Margaret Mead
Luigi Sandon

Posts: 353
Registered: 10/15/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 25, 2014 12:46 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Nobody really NEEDS them. If you want text, one string type is fine. If
you want to store other data, don't use strings.

No. If you want "human readable language" one string type is fine. If you want "computer text", one string type is not enough. Otherwise why do you need many number formats? Store everything into a 64 bit integer, and convert it to and from what you need every time... it's not rocket science. Just a bit waste of time, maybe?

Ah, and no one really need more than one type of drill... after all an hole is always an hole...
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 25, 2014 3:40 AM   in response to: Luigi Sandon in response to: Luigi Sandon
Luigi Sandon wrote:
Nobody really NEEDS them. If you want text, one string type is fine. If
you want to store other data, don't use strings.

No. If you want "human readable language" one string type is fine. If you want "computer text", one string type is not enough. Otherwise why do you need many number formats? Store everything into a 64 bit integer, and convert it to and from what you need every time... it's not rocket science. Just a bit waste of time, maybe?

Ah, and no one really need more than one type of drill... after all an hole is always an hole...

ROFL

Dalija Prasnikar
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 25, 2014 11:36 PM   in response to: Luigi Sandon in response to: Luigi Sandon
Luigi Sandon wrote:

Nobody really NEEDS them. If you want text, one string type is
fine. If you want to store other data, don't use strings.

No. If you want "human readable language" one string type is fine. If
you want "computer text", one string type is not enough.

Again: If you want to store text, one string type is enough. For all
other data, don't use strings. I am not talking about human readable
language. All text is "computer text", and for that, one string type is
fine.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Science is like sex: sometimes something useful comes out, but
that is not the reason we are doing it" -- Richard Feynman
Luigi Sandon

Posts: 353
Registered: 10/15/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 26, 2014 11:46 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Again: If you want to store text, one string type is enough. For all

Again, you think "text" is "human language" - but "computer text" is still text but it's not just one representation of human language. The same way text needs different fonts when displayed (or do you believe a single font is enough for any need?), text may need different formats and encodings when stored and processed.

other data, don't use strings. I am not talking about human readable
language. All text is "computer text", and for that, one string type is
fine.

No. You're completely wrong here. When you say "text" you just mean the semantic meaning of "glyphs" to a human being - not its internal representation for a computer and its software.

And don't worry - never used a string to store something that is not a string. That's why I like pointers and memory buffers - something people like you would remove from Delphi as well, in their attempts to turn it into a scripting language.
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 28, 2014 1:20 PM   in response to: Luigi Sandon in response to: Luigi Sandon
Luigi Sandon wrote:

Again: If you want to store text, one string type is enough. For all

Again, you think "text" is "human language" - but "computer text" is
still text but it's not just one representation of human language.
The same way text needs different fonts when displayed (or do you
believe a single font is enough for any need?), text may need
different formats and encodings when stored and processed.

other data, don't use strings. I am not talking about human readable
language. All text is "computer text", and for that, one string
type is fine.

No. You're completely wrong here. When you say "text" you just mean
the semantic meaning of "glyphs" to a human being - not its internal
representation for a computer and its software.

And don't worry - never used a string to store something that is not
a string. That's why I like pointers and memory buffers - something
people like you would remove from Delphi as well, in their attempts
to turn it into a scripting language.

What a load of horse pucky. Text does not need different encodings. You
are the one who talks about glyphs (what else are fonts?), not I.

There are many modern languages that only have one string type,
generally a Unicode type (UTF-16 or UTF-16, sometimes even UTF-32).

Many of these modern single-string-type programming languages are
extremely good at handling text, and I don't just mean "human
language", I also mean things like JSON, XML, HTML, CSS, Delphi or C++
source code, or even stuff coming in from ports, etc., etc. They handle
strings often much better and more flexible than languages like C++
which have several (too many) string types, usually for historical
reasons. Most of them manage to perform all the tasks C++ or Delphi,
with their too many string types, perform as well.

So no, WE DO NOT NEED MORE THAN ONE STRING TYPE, just like many other
modern languages don't. For some purposes, they may be nice to have, or
can make things easier or faster, but we do not really need them.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Life would be so much easier if we could just see the source
code." -- unknown
Lars Fosdal


Posts: 156
Registered: 10/26/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 25, 2014 1:28 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
OK, it may be a bummer you don't have syntax for complex types or, say,
for decimals, like some other languages, but that does not mean you
can't use them.

Native support could have a significant impact on performance, though.

A proper lib for BCD and arbitrary precision numbers (such as GNU Multiple Precision) would also have been nice.

Off-topic on the string type debate: I'd prefer if there was one string type, and the necessary tools to transform the unicode string to/from a TBytes buffer, with ANSI(CP)/ASCII(CP)/EBCDIC support.
I'd prefer all transforms to be explicit and not implicit, to remove any doubt about what the content actually is.

--
http://plus.lars.fosdal.com
Delphi Developers Google+ Community: https://plus.google.com/communities/103113685381486591754
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 25, 2014 11:40 PM   in response to: Lars Fosdal in response to: Lars Fosdal
Lars Fosdal wrote:

Rudy Velthuis (TeamB) wrote:
OK, it may be a bummer you don't have syntax for complex types or,
say, for decimals, like some other languages, but that does not
mean you can't use them.

Native support could have a significant impact on performance, though.

Perhaps. Not sure how native support would make, say, complex types
more performant. After all, they can be built out of the same floating
point types as the built-in hardware supported types.
--
Rudy Velthuis (TeamB) http://www.teamb.com

"One-tenth of the folks run the world. One-tenth watch them run
it, and the other eighty percent don't know what the hell's
going on."
-- Jake Simmons
Lars Fosdal


Posts: 156
Registered: 10/26/99
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 26, 2014 10:42 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
Lars Fosdal wrote:
Native support could have a significant impact on performance, though.

Perhaps. Not sure how native support would make, say, complex types
more performant. After all, they can be built out of the same floating
point types as the built-in hardware supported types.

http://stackoverflow.com/questions/3211346/complex-mul-and-div-using-sse-instructions
gives examples of optimized complex multiplication and division.

Intel's whitepaper gives performance comparisons
https://software.intel.com/en-us/articles/introduction-to-intel-advanced-vector-extensions

The difference is several orders of magnitude between old school float ops, and SSE/SIMD instructions.

--
http://plus.lars.fosdal.com
Delphi Developers Google+ Community: https://plus.google.com/communities/103113685381486591754
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 28, 2014 1:36 PM   in response to: Lars Fosdal in response to: Lars Fosdal
Lars Fosdal wrote:

Rudy Velthuis (TeamB) wrote:
Lars Fosdal wrote:
Native support could have a significant impact on performance,
though.

Perhaps. Not sure how native support would make, say, complex types
more performant. After all, they can be built out of the same
floating point types as the built-in hardware supported types.

http://stackoverflow.com/questions/3211346/complex-mul-and-div-using-sse-instructions
gives examples of optimized complex multiplication and division.

Yes, but I still don't see how that matters. The SSE instructions use
doubles or floats as the base types. The algorithm can just as well be
put in library code, it does not have to be built-in runtime code.

The ONLY advantage would be a syntactical one, i.e. you would be able
to write

myC := 3.0 + 5.2i;

instead of

myC := Complex(3.0, 5.2);

The library code for the Complex type could just as well implement the
mentioned operators using SSE or whatever turns out to be beneficial.

--
Rudy Velthuis (TeamB) http://www.teamb.com

Weiler's Law: Nothing is impossible for the man who doesn't have
to do the work.

Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 30, 2014 2:05 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 28.06.2014 22:36, schrieb Rudy Velthuis (TeamB):


The ONLY advantage would be a syntactical one, i.e. you would be able
to write

myC := 3.0 + 5.2i;

instead of

myC := Complex(3.0, 5.2);

The library code for the Complex type could just as well implement the
mentioned operators using SSE or whatever turns out to be beneficial.


As is we have neither out of the box, afaik.

Greetings

Markus
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2014 2:22 AM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

The library code for the Complex type could just as well implement
the mentioned operators using SSE or whatever turns out to be
beneficial.

As is we have neither out of the box, afaik.

Not out of the box, but perhaps as 3rd party tool. If there is none,
that would indicate to me that there may not be a big demand for it.
--
Rudy Velthuis (TeamB) http://www.teamb.com

"Five exclamation marks, the sure sign of an insane mind."
-- Terry Pratchett (Reaper Man)
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 25, 2014 3:33 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
Vladimir Ulchenko wrote:

Also give us reduced RTTI and smaller exe's, some applications need
to run on embedded windows with limited memory and the change to
Unicode and RTTI is really making that difficult, an ASCII
stringlist would be lovely.

some people here also have a strange ideas that nobody needs string
types other than unicode

Nobody really NEEDS them. If you want text, one string type is fine. If
you want to store other data, don't use strings.

No it is not, or shall I say it could be enough if internal storage would be
optimized and would not waste memory.

Dalija Prasnikar
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 25, 2014 11:37 PM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Rudy Velthuis (TeamB) wrote:
Vladimir Ulchenko wrote:

Also give us reduced RTTI and smaller exe's, some applications
need to run on embedded windows with limited memory and the
change to Unicode and RTTI is really making that difficult, an
ASCII stringlist would be lovely.

some people here also have a strange ideas that nobody needs
string types other than unicode

Nobody really NEEDS them. If you want text, one string type is
fine. If you want to store other data, don't use strings.

No it is not, or shall I say it could be enough if internal storage
would be optimized and would not waste memory.

Not sure how internal storage of text changes what I say. Internal
storage should be irrelevant.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"A committee is a group of people who individually can do nothing
but together can decide that nothing can be done." -- Fred Allen.
Luigi Sandon

Posts: 353
Registered: 10/15/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 26, 2014 1:00 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Not sure how internal storage of text changes what I say. Internal
storage should be irrelevant.

Again, internal storage is irrelevant to human beings to whom text is displayed. But not to CPU, storage and software... again, why do you use different numeric types and encodings? Why not a single 64 bit numeric type fits all?
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 28, 2014 1:29 PM   in response to: Luigi Sandon in response to: Luigi Sandon
Luigi Sandon wrote:

Not sure how internal storage of text changes what I say. Internal
storage should be irrelevant.

Again, internal storage is irrelevant to human beings to whom text is
displayed.

It should be irrelevant to everyone, just like the internal format of a
Single or Double or Extended should be irrelevant in order to be able
to use them.

If you really need a different format, e.g. to interface with systems
that do not use the same format as the single string type of the
language, you convert, just like you would do when interfacing with
systems that store floats in a different way (e.g. with a different
endianness, or not using IEEE-754, etc.).

If, apart from interfacing with other systems, the internal
representation of the text is relevant to you, something is wrong with
your design. Then you are not treating text as what it is: text, a more
or less opaque type. IOW, you are not writing code in a portable way.

People who have to write C or C++ code that must run on several
different systems know what writing portable code is. The do not rely
on internals of a data type, unless they defined it themselves.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"There are worse things in life than death. Have you ever spent
an evening with an insurance salesman?" -- Woody Allen
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 29, 2014 2:16 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:

People who have to write C or C++ code that must run on several
different systems know what writing portable code is. The do not rely
on internals of a data type, unless they defined it themselves.

We all rely on internals one way or another. There are some internals
we should not rely on, but others are crucial for our work. Encoding and
memory footprint of string data are not internals. You may not care about
them at some point, but there are times when you have to.

Dalija Prasnikar
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 29, 2014 2:42 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

We all rely on internals one way or another.

People writing portable code don't. If they must know the maximum or
minimum of something, or the size, they will use the given limits and
never assume anything. They certainly don't care about internals and
leave it to standard library routines to deal with those.

There are some internals
we should not rely on, but others are crucial for our work. Encoding
and memory footprint of string data are not internals.

No, of course they are not. Not sure how that matters.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"The compulsion to do good is an innate American trait. Only
North Americans seem to believe that they always should, may,
and actually can choose somebody with whom to share their
blessings. Ultimately this attitude leads to bombing people
into the acceptance of gifts."
-- Ivan Illich
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 29, 2014 2:52 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
Dalija Prasnikar wrote:

We all rely on internals one way or another.

People writing portable code don't. If they must know the maximum or
minimum of something, or the size, they will use the given limits and
never assume anything. They certainly don't care about internals and
leave it to standard library routines to deal with those.

I am saying that just like using string type is portable across different
Delphi compilers, so should be UTF8String for instance. There is nothing
special about that type that would not make it portable. I am saying that
decision to remove 8-bit strings was pure marketing decision and not technical
one.

Dalija Prasnikar
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 29, 2014 3:02 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

I am saying that just like using string type is portable across
different Delphi compilers, so should be UTF8String for instance.

It should not matter. If a different format is required, a conversion
should be done.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Whenever I hear anyone arguing for slavery, I feel a strong
impulse to see it tried on him personally."
-- Abraham Lincoln
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 29, 2014 10:59 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy,

| If a different format is required, a conversion
| should be done.

But often a "conversion" is just NOT practical!

--

Q

06/29/2014 10:58:44

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 29, 2014 11:29 AM   in response to: Quentin Correll in response to: Quentin Correll
Quentin Correll wrote:
Rudy,

| If a different format is required, a conversion
| should be done.

But often a "conversion" is just NOT practical!

I am afraid we might be getting into another futile discussion ;-)

Dalija Prasnikar
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 29, 2014 1:43 PM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija,

| | But often a "conversion" is just NOT practical!
| |

| I am afraid we might be getting into another futile discussion ;-)

<chuckling OL> Yeah,... I had the same thought. <g>

--

Q

06/29/2014 13:42:52

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2014 2:20 AM   in response to: Quentin Correll in response to: Quentin Correll
Quentin Correll wrote:

Rudy,

If a different format is required, a conversion
should be done.

But often a "conversion" is just NOT practical!

Then try to avoid conversions until necessary.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Manifest plainness, embrace simplicity, reduce selfishness,
have few desires."
-- Lao tzu
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2014 2:53 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
Quentin Correll wrote:

Rudy,

If a different format is required, a conversion
should be done.

But often a "conversion" is just NOT practical!

Then try to avoid conversions until necessary.

But when you don't have 8-bit strings you cannot avoid conversion,
that's the point. You are forced to do it or you are left to your own
devices handling textual data inside binary data containers.

Dalija Prasnikar
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2014 4:05 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Rudy Velthuis (TeamB) wrote:
Quentin Correll wrote:

Rudy,

If a different format is required, a conversion
should be done.

But often a "conversion" is just NOT practical!

Then try to avoid conversions until necessary.

But when you don't have 8-bit strings you cannot avoid conversion,
that's the point.

Depends on what you intend to do with the data. And I wrote "until
necessary", not "forever".

Again, all the things you guys do with data are very likely done by
others too, and much of this in languages which only have one string
type. They probably also receive single-byte texts, etc. and probably
have to convert. But they seem to manage very-well-thank-you, so what
is wrong with Delphi developers that they can't?
--
Rudy Velthuis (TeamB) http://www.teamb.com

"I'm fed up to the ears with old men dreaming up wars for
young men to die in." -- George McGovern
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2014 4:32 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
Dalija Prasnikar wrote:

Rudy Velthuis (TeamB) wrote:
Quentin Correll wrote:

Rudy,

If a different format is required, a conversion
should be done.

But often a "conversion" is just NOT practical!

Then try to avoid conversions until necessary.

But when you don't have 8-bit strings you cannot avoid conversion,
that's the point.

Depends on what you intend to do with the data. And I wrote "until
necessary", not "forever".

And what when you have situation where conversion should never occur?
I am processing large UTF8 encoded XML files, where they can stay in UTF8
format during whole processing. Converting them back and forth is just
exercise in futility.

Again, all the things you guys do with data are very likely done by
others too, and much of this in languages which only have one string
type. They probably also receive single-byte texts, etc. and probably
have to convert. But they seem to manage very-well-thank-you, so what
is wrong with Delphi developers that they can't?

Will you please stop pushing what other languages have and don't have.
It is not relevant. There are also popular languages that have more than
one string type.

It is not that something was wrong with us, but something was taken away
from us.

Dalija Prasnikar
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2014 11:44 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija,

| It is not that something was wrong with us, but something was taken
| away from us.

Damn straight!

--

Q

07/02/2014 11:43:52

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2014 2:15 PM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

And what when you have situation where conversion should never occur?
I am processing large UTF8 encoded XML files, where they can stay in UTF8
format during whole processing. Converting them back and forth is just
exercise in futility.

What do you convert that UTF8 text/data to and from?
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 3, 2014 7:20 AM   in response to: Adem Meda in response to: Adem Meda
Adem Meda wrote:
Dalija Prasnikar wrote:

And what when you have situation where conversion should never occur?
I am processing large UTF8 encoded XML files, where they can stay in UTF8
format during whole processing. Converting them back and forth is just
exercise in futility.

What do you convert that UTF8 text/data to and from?

I don't convert it, I leave it in UTF8 format and do direct processing. If I would
use Delphi unicode string type I would had two unneccessary conversions that
take time + doubled memory requirements (that would mean that processing is
no longer possible on 32-bit Windows I still have to support).

Dalija Prasnikar
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 3, 2014 6:17 PM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

What do you convert that UTF8 text/data to and from?

I don't convert it, I leave it in UTF8 format and do direct processing. If I
would use Delphi unicode string type I would had two unneccessary conversions
that take time + doubled memory requirements (that would mean that processing
is no longer possible on 32-bit Windows I still have to support).

(Disclaimer: Even though I work with a lot of text data, I don't do mobile.)

I don't know what sort of applications you deal with that doubling (or even
quadrubling) the memory used by text data means prohibitively high usage of
RAM, or makes the app too slow.

But personally I have never come across a case where that sort of thing had
any significant effect on my apps.
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 1:33 AM   in response to: Adem Meda in response to: Adem Meda
Adem Meda wrote:
Dalija Prasnikar wrote:

What do you convert that UTF8 text/data to and from?

I don't convert it, I leave it in UTF8 format and do direct processing. If I
would use Delphi unicode string type I would had two unneccessary conversions
that take time + doubled memory requirements (that would mean that processing
is no longer possible on 32-bit Windows I still have to support).

(Disclaimer: Even though I work with a lot of text data, I don't do mobile.)

I don't know what sort of applications you deal with that doubling (or even
quadrubling) the memory used by text data means prohibitively high usage of
RAM, or makes the app too slow.

Try loading 500MB XML in memory and do something with it. With UTF8 it will
consume 500MB and another 500MB for changes, double that and you are out
of memory real soon.

Dalija Prasnikar
Matthew Jones

Posts: 337
Registered: 1/25/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 2:44 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Try loading 500MB XML in memory and do something with it. With UTF8
it will consume 500MB and another 500MB for changes, double that and
you are out of memory real soon.

I have a project that a customer complained it was failing, and it was
trying to copy a 2GB XML file "in transit". The server already had the
/3GB option set, but I had to modify the code to ensure it wasn't
copying it. This was actually in a memory stream, but at some points
the data could be interpreted as XML (indeed, at the far end of the
transit it is opened and decoded).

It was never expected to handle such large XML, but I've found that
these things creep up on you. Mobile computers are not far off the
desktop of a few years ago, only they have less RAM. Why would anyone
design a mobile development system that automatically doubled the
memory requirements?

Matthew
Asbjørn Heid

Posts: 267
Registered: 11/12/12
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 2:53 AM   in response to: Matthew Jones in response to: Matthew Jones
Matthew Jones wrote:
It was never expected to handle such large XML, but I've found that
these things creep up on you.

Indeed. I've worked on an open-source rendering program. Didn't take long before we got reports that users were running out of memory trying to render images over 10000x10000 pixels large (think big poster at 300 DPI). For example I had to re-organize some code to perform two loops over the same data filling two buffers separately, so I could free/allocate the buffers inbetween, instead of looping only once but holding both buffers in memory at once.

Also had to write a more clever memory stream object which used a sequence of blocks, each 64 MB, instead of one contiguous allocation for our 32bit users so they could push it a bit further and not get limited by memory fragmentation.

So yes, these things often creep up on you.

- Asbjørn
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 4:05 AM   in response to: Asbjørn Heid in response to: Asbjørn Heid
Asbjørn Heid wrote:
Matthew Jones wrote:
It was never expected to handle such large XML, but I've found that
these things creep up on you.

Indeed. I've worked on an open-source rendering program. Didn't take long before we got reports that users were running out of memory trying to render images over 10000x10000 pixels large (think big poster at 300 DPI). For example I had to re-organize some code to perform two loops over the same data filling two buffers separately, so I could free/allocate the buffers inbetween, instead of looping only once but holding both buffers in memory at once.

Also had to write a more clever memory stream object which used a sequence of blocks, each 64 MB, instead of one contiguous allocation for our 32bit users so they could push it a bit further and not get limited by memory fragmentation.

So yes, these things often creep up on you.

Yes, they do. And if you have to workaround such issues your
code becames way more complex.

Dalija Prasnikar
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 5:57 PM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Yes, they do. And if you have to workaround such issues your
code becames way more complex.

Oh, those bloody customers and their unending demands.. wouldn't life be a lot
simpler if stopped or ceased to exist? :)
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 8:23 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Asbjørn Heid wrote:
Matthew Jones wrote:
It was never expected to handle such large XML, but I've found
that these things creep up on you.

Indeed. I've worked on an open-source rendering program. Didn't
take long before we got reports that users were running out of
memory trying to render images over 10000x10000 pixels large (think
big poster at 300 DPI). For example I had to re-organize some code
to perform two loops over the same data filling two buffers
separately, so I could free/allocate the buffers inbetween, instead
of looping only once but holding both buffers in memory at once.

Also had to write a more clever memory stream object which used a
sequence of blocks, each 64 MB, instead of one contiguous
allocation for our 32bit users so they could push it a bit further
and not get limited by memory fragmentation.

So yes, these things often creep up on you.

Yes, they do. And if you have to workaround such issues your
code becames way more complex.

Not necessarily. Write a class that takes care of the loading and
caching (or let the OS do it for you) and uses memory mapped files or
similar techniques.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Five exclamation marks, the sure sign of an insane mind."
-- Terry Pratchett (Reaper Man)
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 12:49 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 05.07.2014 17:23, schrieb Rudy Velthuis (TeamB):


Yes, they do. And if you have to workaround such issues your
code becames way more complex.

Not necessarily. Write a class that takes care of the loading and
caching (or let the OS do it for you) and uses memory mapped files or
similar techniques.

That doesn't away with the issue, that customers sometimes simply demand
things which are either not doable or do not fit well with the
architecture the program the want enhanced is bsed on. ;-)

Greetings

Markus
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 3:58 PM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

That doesn't away with the issue, that customers sometimes simply demand
things which are either not doable or do not fit well with the
architecture the program the want enhanced is bsed on. ;-)

And, you expect 'string' to offer the ideal solution to customers whims?
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 2:19 AM   in response to: Adem Meda in response to: Adem Meda
Am 06.07.2014 00:58, schrieb Adem Meda:
Markus Humm wrote:

That doesn't away with the issue, that customers sometimes simply demand
things which are either not doable or do not fit well with the
architecture the program the want enhanced is bsed on. ;-)

And, you expect 'string' to offer the ideal solution to customers whims?

For some of those yes. ;-)
(btw. that question can be asked for all nearly language features, so
better stop asking in such ways!)

Greetings

Markus
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 11:44 AM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

Am 05.07.2014 17:23, schrieb Rudy Velthuis (TeamB):


Yes, they do. And if you have to workaround such issues your
code becames way more complex.

Not necessarily. Write a class that takes care of the loading and
caching (or let the OS do it for you) and uses memory mapped files
or similar techniques.

That doesn't away with the issue, that customers sometimes simply
demand things which are either not doable or do not fit well with the
architecture the program the want enhanced is bsed on. ;-)

Huh? The issue was that if you have to workaround issues of memory
management (i.e. using memory mapped files or similar techniques) the
code gets more complex. Not IME.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"We totally deny the allegations, and we are trying to identify
the allegators."
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 8:21 AM   in response to: Asbjørn Heid in response to: Asbjørn Heid
Asbjørn Heid wrote:

Matthew Jones wrote:
It was never expected to handle such large XML, but I've found that
these things creep up on you.

Indeed. I've worked on an open-source rendering program. Didn't take
long before we got reports that users were running out of memory
trying to render images over 10000x10000 pixels large (think big
poster at 300 DPI).

Heck, and that problem was not even caused by the use of a certain
string type. <g>

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Be kind; everyone you meet is fighting a hard battle."
-- Plato

The enemy is anybody who's going to get you killed, no matter
which side he is on."
-- Joseph Heller

Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 9:57 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy,

| Heck, and that problem was not even caused by the use of a certain
| string type. <g>

LOL!

--

Q

07/05/2014 09:57:27

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 8:20 AM   in response to: Matthew Jones in response to: Matthew Jones
Matthew Jones wrote:

Why would anyone design a mobile development system that
automatically
doubled the memory requirements?

Doubled? Not if you compare it to the alternatives. Note that if you
programmed in Java, C# (on MonoXXX), Objective-C + Cocoa or even Swift
+ Cocoa, the memory requirements would be the same, if not more, since
all these use UTF-16 as (single) string type.
--
Rudy Velthuis (TeamB) http://www.teamb.com

Mencken's Law: There is always an easy answer to every human
problem -- neat, plausible, and wrong.
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 12:52 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 05.07.2014 17:20, schrieb Rudy Velthuis (TeamB):
Matthew Jones wrote:

Why would anyone design a mobile development system that
automatically
doubled the memory requirements?

Doubled? Not if you compare it to the alternatives. Note that if you
programmed in Java, C# (on MonoXXX), Objective-C + Cocoa or even Swift
+ Cocoa, the memory requirements would be the same, if not more, since
all these use UTF-16 as (single) string type.

Hello,

and? That's no reason for throwing the already existing types away
completely which would just be a less memory "hungry" alternative for
those scenarioius which need it. Those scenarious are there and afaik
most of the string routines were there in pure pascal versions already
anyway. (means: they could have left over one of those and only clean up
the others)

Greetings

Markus
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 11:44 AM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

Am 05.07.2014 17:20, schrieb Rudy Velthuis (TeamB):
Matthew Jones wrote:

Why would anyone design a mobile development system that
automatically
doubled the memory requirements?

Doubled? Not if you compare it to the alternatives. Note that if you
programmed in Java, C# (on MonoXXX), Objective-C + Cocoa or even
Swift + Cocoa, the memory requirements would be the same, if not
more, since all these use UTF-16 as (single) string type.

Hello,

and? That's no reason for throwing the already existing types away
completely which would just be a less memory "hungry" alternative for
those scenarioius which need it.

They were not "thrown away", they were simply not carried over, for the
sake of cleaning up the mess that exists on Windows.

And unless you do things like Dalijah describes, the size of the string
elements does not play a big role, especially not in mobile apps, since
you would be extremely foolish to keep much of them in memory. On a
mobile platform, you probably won't do any processing of larger XML
files, or extensive DB apps, etc. At least not yet.
--
Rudy Velthuis (TeamB) http://www.teamb.com

"Life would be so much easier if we could just see the source
code." -- unknown
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 15, 2014 12:56 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 06.07.2014 20:44, schrieb Rudy Velthuis (TeamB):
Markus Humm wrote:

Am 05.07.2014 17:20, schrieb Rudy Velthuis (TeamB):
Matthew Jones wrote:

Why would anyone design a mobile development system that
automatically
doubled the memory requirements?

Doubled? Not if you compare it to the alternatives. Note that if you
programmed in Java, C# (on MonoXXX), Objective-C + Cocoa or even
Swift + Cocoa, the memory requirements would be the same, if not
more, since all these use UTF-16 as (single) string type.

Hello,

and? That's no reason for throwing the already existing types away
completely which would just be a less memory "hungry" alternative for
those scenarioius which need it.

They were not "thrown away", they were simply not carried over, for the
sake of cleaning up the mess that exists on Windows.

And unless you do things like Dalijah describes, the size of the string
elements does not play a big role, especially not in mobile apps, since
you would be extremely foolish to keep much of them in memory. On a
mobile platform, you probably won't do any processing of larger XML
files, or extensive DB apps, etc. At least not yet.

Hello,

seems like the first time you're admitting that there are cases where
these things matter.

And one could argue if they were "thrown away" or not. As a language
specification already existed and there was no real need to change that
specification in that area to be able to create a mobile compiler.

Greetings

Markus
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 15, 2014 3:05 PM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

Hello,

seems like the first time you're admitting that there are cases where
these things matter.

Such things matter to those whose code relies on internals, like if
text is stored as UTF-8 or UTF-16, as a single, contiguous array or
some other structure (trie?), etc. indeed. We have already seen that
code relying on such internals loses portability and may not work
everywhere.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Always do right- this will gratify some and astonish the rest."
-- Mark Twain (1835-1910)
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 16, 2014 1:16 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 16.07.2014 00:05, schrieb Rudy Velthuis (TeamB):
Markus Humm wrote:

Hello,

seems like the first time you're admitting that there are cases where
these things matter.

Such things matter to those whose code relies on internals, like if
text is stored as UTF-8 or UTF-16, as a single, contiguous array or
some other structure (trie?), etc. indeed. We have already seen that
code relying on such internals loses portability and may not work
everywhere.

In most cases you are of course right, but as you present is usually is
"I'm right in 100% of cases". But: there are cases where highest speed
is necessary and thus it cannot be properly avoided to code against some
internals. If memory is rare and data is in a 8-bit string compatible
format anyway using of utf-16 based strings would be a wasteful and thus
wrong approach for isntance.

Greetings

Markus
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 19, 2014 11:43 AM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

In most cases you are of course right, but as you present is usually
is "I'm right in 100% of cases". But: there are cases where highest
speed is necessary and thus it cannot be properly avoided to code
against some internals.

Yes, these are hacks and they may sometimes be necessary, indeed. They
are usually not portable, and may even break in the next version of the
compiler or platform. Anyone using such hacks should be aware of that.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"The most overlooked advantage of owning a computer is that if
they foul up there's no law against whacking them around a bit."
-- Eric Porterfield.
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 3:36 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Adem Meda wrote:
Dalija Prasnikar wrote:

What do you convert that UTF8 text/data to and from?

I don't convert it, I leave it in UTF8 format and do direct processing.
If I would use Delphi unicode string type I would had two unneccessary
conversions that take time + doubled memory requirements (that would mean
that processing is no longer possible on 32-bit Windows I still have to
support).

(Disclaimer: Even though I work with a lot of text data, I don't do mobile.)

I don't know what sort of applications you deal with that doubling (or even
quadrubling) the memory used by text data means prohibitively high usage of
RAM, or makes the app too slow.

Try loading 500MB XML in memory and do something with it. With UTF8 it will
consume 500MB and another 500MB for changes, double that and you are out
of memory real soon.

Even though 500 MB is peanuts for my hardware, let's (for the sake of
discussion) consider it too big to load into RAM: Why would I load it into RAM
and why not use memory mapped files?

Plus, if we're talking about XML, I hope you don't use a DOM tree version.
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 4:08 AM   in response to: Adem Meda in response to: Adem Meda
Adem Meda wrote:
Dalija Prasnikar wrote:

Adem Meda wrote:
Dalija Prasnikar wrote:

What do you convert that UTF8 text/data to and from?

I don't convert it, I leave it in UTF8 format and do direct processing.
If I would use Delphi unicode string type I would had two unneccessary
conversions that take time + doubled memory requirements (that would mean
that processing is no longer possible on 32-bit Windows I still have to
support).

(Disclaimer: Even though I work with a lot of text data, I don't do mobile.)

I don't know what sort of applications you deal with that doubling (or even
quadrubling) the memory used by text data means prohibitively high usage of
RAM, or makes the app too slow.

Try loading 500MB XML in memory and do something with it. With UTF8 it will
consume 500MB and another 500MB for changes, double that and you are out
of memory real soon.

Even though 500 MB is peanuts for my hardware, let's (for the sake of
discussion) consider it too big to load into RAM: Why would I load it into RAM
and why not use memory mapped files?

It is not peanuts on 32-bit Windows. And strings make that code simple, clean and
easy to understand and change. Anything else would make that code more complex.

Plus, if we're talking about XML, I hope you don't use a DOM tree version.

Nope. I don't have to create tree at all.

Dalija Prasnikar
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 8:55 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Adem Meda wrote:
Even though 500 MB is peanuts for my hardware, let's (for the sake of
discussion) consider it too big to load into RAM: Why would I load it into
RAM and why not use memory mapped files?

It is not peanuts on 32-bit Windows.

Good thing, then, that memory-mapping is also available in 32-bit Windows; if
only people concerned about memory would use it ;)

And strings make that code simple, clean and easy to understand and
change. Anything else would make that code more complex.

If you can (simply by looking at the stream) remove the BOM header and then
decipher each variable-length byte chain (UTF8 is just that, after all), and
promptly visualize the font glyph they represent, then, yes, working directly
with UTF8 strings is simple and clean..

Plus, if we're talking about XML, I hope you don't use a DOM tree version.

Nope. I don't have to create tree at all.

Releived to hear that.

You had me, however, worried there for a second ;)
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 11:28 AM   in response to: Adem Meda in response to: Adem Meda
Adem Meda wrote:
Dalija Prasnikar wrote:

Adem Meda wrote:
Even though 500 MB is peanuts for my hardware, let's (for the sake of
discussion) consider it too big to load into RAM: Why would I load it into
RAM and why not use memory mapped files?

It is not peanuts on 32-bit Windows.

Good thing, then, that memory-mapping is also available in 32-bit Windows; if
only people concerned about memory would use it ;)

If simpler code can do the job and has been doing job for ages on even less
powerfull computers than we have today, how can I explain that I need to
spend valuable time modifying working code just because my development tool
got crappier.

And strings make that code simple, clean and easy to understand and
change. Anything else would make that code more complex.

If you can (simply by looking at the stream) remove the BOM header and then
decipher each variable-length byte chain (UTF8 is just that, after all), and
promptly visualize the font glyph they represent, then, yes, working directly
with UTF8 strings is simple and clean..

I don't need to interpret any characters above 127, and I don't need to show any
data during processing.

Dalija Prasnikar
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 5:57 PM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

If simpler code can do the job and has been doing job for ages on even less
powerfull computers than we have today, how can I explain that I need to
spend valuable time modifying working code just because my development tool
got crappier.

Could it be because you started off wrong? I mean, we all abused strings as
in-RAM data storage but to claim that it is a right to continue to do so is,
IMO, strecthing it a bit too far.

I don't need to interpret any characters above 127, and I don't need to show
any data during processing.

There, then, you already have the opportunity to reduce the memory footprint of
your 200 MB XML string by (probably more than) 50 % if only you read it into
memory as 2 chars per byte.

In short: At least for the example you gave, I can't find a reason to
sympathise with your complaints when you're reading the whole file into RAM in
one go which you shouldn't; and, use 'string' for it when you really should use
a more suitable structure. What's worse is that implementing any of this is not
at all complicated.
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 1:24 AM   in response to: Adem Meda in response to: Adem Meda
Adem Meda wrote:
Dalija Prasnikar wrote:

If simpler code can do the job and has been doing job for ages on even less
powerfull computers than we have today, how can I explain that I need to
spend valuable time modifying working code just because my development tool
got crappier.

Could it be because you started off wrong? I mean, we all abused strings as
in-RAM data storage but to claim that it is a right to continue to do so is,
IMO, strecthing it a bit too far.

So? I started it wrong. Yes, it would be easy to blame it on me. When I started
using Delphi, I had to explain why we are using it instead of C++ like all other
"normal" developers do. I showed how we can make things faster and easier with
Delphi, and strings were responsible for significant part of that faster and easier.

I don't care if my code was wrong, or I abused something or not. People abused
strings because it made their life easier. I am not going to change my decision
to base everything around UTF8 encodings because that was right design decision.
Even if I would be using memory mapped files I would still work with UTF8 strings.
Period.

I don't need to interpret any characters above 127, and I don't need to show
any data during processing.

There, then, you already have the opportunity to reduce the memory footprint of
your 200 MB XML string by (probably more than) 50 % if only you read it into
memory as 2 chars per byte.

What are you talking about here. How can I read it as two chars per byte when I have
only single bit left???

In short: At least for the example you gave, I can't find a reason to
sympathise with your complaints when you're reading the whole file into RAM in
one go which you shouldn't; and, use 'string' for it when you really should use
a more suitable structure. What's worse is that implementing any of this is not
at all complicated.

What more suitable structure? Array of bytes, for textual data? Where I don't have single
processing function available out of the box.

Dalija Prasnikar
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 2:00 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

What are you talking about here. How can I read it as two chars per byte when I have
only single bit left???


Ok, I am starting to write stupidities here ;-)

Anyway, I don't need to compact data further, and that would require even more work
on my side to write all necessary support code.

Dalija Prasnikar
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 2:08 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:
Dalija Prasnikar wrote:

What are you talking about here. How can I read it as two chars per byte when I have
only single bit left???


Ok, I am starting to write stupidities here ;-)

Or not ;-)

Dalija Prasnikar
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 2:14 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Adem Meda wrote:

Could it be because you started off wrong? I mean, we all abused strings as
in-RAM data storage but to claim that it is a right to continue to do so is,
IMO, strecthing it a bit too far.

So? I started it wrong. Yes, it would be easy to blame it on me.

I am not blaming it on you. On the contrary, if anything, I am blaming it on
all of us: All the way from language designers to the coders and everything in
between.

But, we (the whole chain, that is) just didn't know what the future could bring
up or didn't care enough.

This is happening in almost all languages; especially those that started out as
a way to code in fast and simple ways. Well, it turns out, as the needs to
cover wider scopes, those fast and simple langugaes are no more enough; and
that they have to introduce breaking changes. Almost all scripting languages
suffered (or will suffer) from this headache; PHP, Perl, Phyton etc.

Heck, even Java has had (subtle or not) breaking changes that some people are
scared off of compiling with newer versions.

So, what I am trying to say is this: It's not you. It is the rest of world plus
you. :)

When I started using Delphi, I had to explain why we are using it instead
of C++ like all other "normal" developers do. I showed how we can make
things faster and easier with Delphi, and strings were responsible
for significant part of that faster and easier.

OK. So those were the good times.

We all enjoyed it while they lasted.

Now, times are achanging --again.

And, it really is a shame (or, tough luck) that one of the things that are
being affected is strings in Delphi (the other big one one, I suppose, is our
initial assumptions of SizeOf(Pointer) or SizeOf(Char) ).

But, we have to be as realistic as possible and see the writing on the wall: No
amount of complaints will reverse this course much --entropy has taken over.

On the subject of entropy <g> here is a few lines written by someone (Omar
Khayyam) that I enjoy quite a bit:

The Moving Finger writes and having writ,
Moves on; nor all your piety nor wit
Shall lure it back to cancel half a line,
Nor all your tears blot out a word of it.

Yeah.. instead of struggling, try to embrace the changes and adapt; or the
changes will embrace and suffocate you.

I don't care if my code was wrong, or I abused something or not. People
abused strings because it made their life easier. I am not going to change my
decision to base everything around UTF8 encodings because that was right
design decision. Even if I would be using memory mapped files I would still
work with UTF8 strings. Period.

Well.. you will, but aparently not without an unreasonable amount of
suffering in the process. Sad thing is, you will induce much of onto yourself
due to all the struggle you'll put out.

I don't need to interpret any characters above 127, and I don't need to
show any data during processing.

There, then, you already have the opportunity to reduce the memory
footprint of your 200 MB XML string by (probably more than) 50 % if only
you read it into memory as 2 chars per byte.

What are you talking about here. How can I read it as two chars per byte when
I have only single bit left???

A byte can hold values in the range of 0..255; right?

So, what you do is you splice that byte into 2 so that you have 2 halves one
ranging from 0..127 and the other ranging from 128..255. IOW, you need o do a
bitwise operation (SHL, SHR stuff) --ask about this in e.p.d.l.delphi.general.

So, in your specialized case, an array of byte can hold (at least) twice as
much information as UTF8String.

Actually, you might even get 6 or 7 chars per byte if all you are interested in
is case-insensitive alphanumerics (plus a few symbols).

In short: At least for the example you gave, I can't find a reason to
sympathise with your complaints when you're reading the whole file into RAM
in one go which you shouldn't; and, use 'string' for it when you really
should use a more suitable structure. What's worse is that implementing any
of this is not at all complicated.

What more suitable structure? Array of bytes, for textual data? Where I don't
have single processing function available out of the box.

As much as I criticise Rudy for other things, I believe he did have the correct
solution that we need to use libraries (IOW, specialized code to handle our
specialized needs) but unfortunately he was shouted down mostly on the grounds
that his solution would disturb people's comfort zones --not because he was
wrong.

A shame, IMO.

What I am trying to say is this: You really shouldn't expect shrink-wrapped
solutions to your specialized needs from the language/compiler as it will
simply mean adding bloat to already quite bloated environment.

You should use/write snippets of code (and collect them in a library, if you
like) to solve a problem like the one you described.

It would be, IMO, the wiser choice especially when compared to strugling and
complaining for things likely never to arrive (not in time, anyway).
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 3:05 AM   in response to: Adem Meda in response to: Adem Meda
Adem Meda wrote:
Dalija Prasnikar wrote:

Adem Meda wrote:

Could it be because you started off wrong? I mean, we all abused strings as
in-RAM data storage but to claim that it is a right to continue to do so is,
IMO, strecthing it a bit too far.

So? I started it wrong. Yes, it would be easy to blame it on me.

I am not blaming it on you. On the contrary, if anything, I am blaming it on
all of us: All the way from language designers to the coders and everything in
between.

But, we (the whole chain, that is) just didn't know what the future could bring
up or didn't care enough.

That is exactly what I am trying to dispute here. It is not about that we didin't know
what future will bring, it is that there is absolutely no reason to remove 8-bit strings
from language that already has them. They have purpose and usability that goes
beyond mere legacy code support.

My code has been made fully Unicode string compatible with Delphi 2010, and I
anticipated Unicode changes well before. I used UTF8String as base type even
in Delphi 7.

8-bit strings are the best tool for the job, moreover they are proper tool for the job.
I am not going to bend over backwards to do things differently just because someone
thinks differently.

This is happening in almost all languages; especially those that started out as
a way to code in fast and simple ways. Well, it turns out, as the needs to
cover wider scopes, those fast and simple langugaes are no more enough; and
that they have to introduce breaking changes. Almost all scripting languages
suffered (or will suffer) from this headache; PHP, Perl, Phyton etc.

Heck, even Java has had (subtle or not) breaking changes that some people are
scared off of compiling with newer versions.

So, what I am trying to say is this: It's not you. It is the rest of world plus
you. :)

I can understand breaking changes when they are meaningfull.

When I started using Delphi, I had to explain why we are using it instead
of C++ like all other "normal" developers do. I showed how we can make
things faster and easier with Delphi, and strings were responsible
for significant part of that faster and easier.

OK. So those were the good times.

We all enjoyed it while they lasted.

Now, times are achanging --again.

Yeah, and now C++ is back on the table. Something I would never ever thought
it would happen. I only hope I have enough strength to fight off such ideas, and
that EMB doesn't screw up desktop compiler.

And, it really is a shame (or, tough luck) that one of the things that are
being affected is strings in Delphi (the other big one one, I suppose, is our
initial assumptions of SizeOf(Pointer) or SizeOf(Char) ).

I have never made such assumptions. Being here from TP days, I knew what
might change and I did my best to avoid writing such code. Or at least to confine it
in some fixed place where I can easily change it.


Yeah.. instead of struggling, try to embrace the changes and adapt; or the
changes will embrace and suffocate you.

I am not struggling at all. The only struggle is trying to convince EMB that removing
8-bit strings was a mistake that should be corrected.

I don't care if my code was wrong, or I abused something or not. People
abused strings because it made their life easier. I am not going to change my
decision to base everything around UTF8 encodings because that was right
design decision. Even if I would be using memory mapped files I would still
work with UTF8 strings. Period.

Well.. you will, but aparently not without an unreasonable amount of
suffering in the process. Sad thing is, you will induce much of onto yourself
due to all the struggle you'll put out.

No I will not. Not for desktop apps.

I don't need to interpret any characters above 127, and I don't need to
show any data during processing.

There, then, you already have the opportunity to reduce the memory
footprint of your 200 MB XML string by (probably more than) 50 % if only
you read it into memory as 2 chars per byte.

What are you talking about here. How can I read it as two chars per byte when
I have only single bit left???

A byte can hold values in the range of 0..255; right?

So, what you do is you splice that byte into 2 so that you have 2 halves one
ranging from 0..127 and the other ranging from 128..255. IOW, you need o do a
bitwise operation (SHL, SHR stuff) --ask about this in e.p.d.l.delphi.general.

I was doing bit compressions more than 20 years ago. That is what my initial
instinct told me (and it was right), you cannot add values because you cannot
get back original characters. If I need to represent 127 values I need 7 bits for
each character.

So, in your specialized case, an array of byte can hold (at least) twice as
much information as UTF8String.

Actually, you might even get 6 or 7 chars per byte if all you are interested in
is case-insensitive alphanumerics (plus a few symbols).

That is another issue here, while I mostly do have characters that fall into 7-bits
I also have others so I would need to reinvent my own codepoints.

In short: At least for the example you gave, I can't find a reason to
sympathise with your complaints when you're reading the whole file into RAM
in one go which you shouldn't; and, use 'string' for it when you really
should use a more suitable structure. What's worse is that implementing any
of this is not at all complicated.

What more suitable structure? Array of bytes, for textual data? Where I don't
have single processing function available out of the box.

As much as I criticise Rudy for other things, I believe he did have the correct
solution that we need to use libraries (IOW, specialized code to handle our
specialized needs) but unfortunately he was shouted down mostly on the grounds
that his solution would disturb people's comfort zones --not because he was
wrong.

I have library, and my library uses UTF8String and that is how it is going to stay.


What I am trying to say is this: You really shouldn't expect shrink-wrapped
solutions to your specialized needs from the language/compiler as it will
simply mean adding bloat to already quite bloated environment.

I don't expect shrink wrapped solutions, I have shrink wrapped solution, for desktop
anyway, and as far as mobile is concerned EMB will either provide me with
what I need or I will not use their solution (BTW, 8-bit strings are not the only
showstopper here)

It would be, IMO, the wiser choice especially when compared to strugling and
complaining for things likely never to arrive (not in time, anyway).

I have big mouth, so I complain, but at the end I am far beyond caring in this matter.
I have moved along a long time ago. If I would see that I am the only idiot that complains
about this I would stop. But I am not the only one with such POV.

Dalija Prasnikar
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 7:26 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

That is exactly what I am trying to dispute here. It is not about that we
didin't know what future will bring, it is that there is absolutely no reason
to remove 8-bit strings from language that already has them. They have
purpose and usability that goes beyond mere legacy code support.

'String' has too much historical contamination and baggage for it to be
salvaged.

I'd rather have an three different types of array of 'codepoint's (i.e. 1-byte,
2-byte, 4-byte 'codepoint's); zero-based and reference counted.

My code has been made fully Unicode string compatible with Delphi 2010, and I
anticipated Unicode changes well before. I used UTF8String as base type even
in Delphi 7.

And I never agreed with UTF8String as a fundamental type. It is simply another
form of 1-bye string except that chars are byte-wise variable length.

8-bit strings are the best tool for the job, moreover they are proper tool
for the job. I am not going to bend over backwards to do things differently
just because someone thinks differently.

I don't agree. 8-bit string is a historical accident at best. As an array of
'char' they are inconsistent with the rest of arrays; whatever human-readable
text they are supposed to hold depends on the codepage yet it itself does not
contain any codepage information.

Actually, you might even get 6 or 7 chars per byte if all you are
interested in is case-insensitive alphanumerics (plus a few symbols).

That is another issue here, while I mostly do have characters that fall into
7-bits I also have others so I would need to reinvent my own codepoints.

Well.. I have had to do just that a few times. When I work with a certain
subset of characters (my own alphabet, so to speak), it makes life simpler
sometimes to invent your own 'codepoint's.

I have big mouth, so I complain, but at the end I am far beyond caring in
this matter. I have moved along a long time ago. If I would see that I am
the only idiot that complains about this I would stop. But I am not the only
one with such POV.

Everyone has different reasons and their reasons differ by how they abused
strings in the past. IOW, not a very coherent crowd.
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 10:24 AM   in response to: Adem Meda in response to: Adem Meda
Adem Meda wrote:
Dalija Prasnikar wrote:

That is exactly what I am trying to dispute here. It is not about that we
didin't know what future will bring, it is that there is absolutely no reason
to remove 8-bit strings from language that already has them. They have
purpose and usability that goes beyond mere legacy code support.

'String' has too much historical contamination and baggage for it to be
salvaged.

I disagree.


8-bit strings are the best tool for the job, moreover they are proper tool
for the job. I am not going to bend over backwards to do things differently
just because someone thinks differently.

I don't agree. 8-bit string is a historical accident at best. As an array of
'char' they are inconsistent with the rest of arrays; whatever human-readable
text they are supposed to hold depends on the codepage yet it itself does not
contain any codepage information.

8-bit strings in Unicode versions of Delphi contain codepages.

Everyone has different reasons and their reasons differ by how they abused
strings in the past. IOW, not a very coherent crowd.

I abused nothing. I used strings for exact purpose they have been made for.

Dalija Prasnikar
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 2:44 PM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

I abused nothing. I used strings for exact purpose they have been made for.

Of course you didn't abuse a thing.

No one does.

No one did.

Loading hundreds of megabytes of XML data into a string is definitely NOT an
abuse.

What's more, you have every right to do just that under a 32bit OS.

And, if the OS (plus Delphi) does not take magical measures against being
bogged down, it is also your right to complain as loudly as you wish.

I fully sympathise wit all that.
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 2:21 AM   in response to: Adem Meda in response to: Adem Meda
Am 05.07.2014 23:44, schrieb Adem Meda:
Dalija Prasnikar wrote:

I abused nothing. I used strings for exact purpose they have been made for.

Of course you didn't abuse a thing.

No one does.

No one did.

Loading hundreds of megabytes of XML data into a string is definitely NOT an
abuse.

What's more, you have every right to do just that under a 32bit OS.

And, if the OS (plus Delphi) does not take magical measures against being
bogged down, it is also your right to complain as loudly as you wish.

I fully sympathise wit all that.

Hello,

don't be so sarcastic. You do not know what that application is for and
what the user base and their expectation regarding this application is.

Greetings

Markus
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 6:44 AM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

don't be so sarcastic. You do not know what that application is for and
what the user base and their expectation regarding this application is.

Of course, and obviously, I don't know what the exact application is --other
than the fact that it loads 200 MB XML file into a string.

But, I can see uour point: I need to be more careful about being politically
correct.
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 6:58 AM   in response to: Adem Meda in response to: Adem Meda
Adem Meda wrote:
Markus Humm wrote:

don't be so sarcastic. You do not know what that application is for and
what the user base and their expectation regarding this application is.

Of course, and obviously, I don't know what the exact application is --other
than the fact that it loads 200 MB XML file into a string.

And it works perfectly. It would not work if I would have to change my library
to use UTF16 strings instead of UTF8.

Dalija Prasnikar
david hoke

Posts: 616
Registered: 2/9/07
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 7:01 AM   in response to: Adem Meda in response to: Adem Meda
"Adem Meda" <adem dot meda at gmail dot com> wrote in message
news:684606 at forums dot embarcadero dot com...

And I never agreed with UTF8String as a fundamental type. It is simply
another
form of 1-bye string except that chars are byte-wise variable length.

hehe, you mean unit-wise variable length, same as Unicode strings in windows
these days...?
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 10:41 AM   in response to: david hoke in response to: david hoke
david hoke wrote:

"Adem Meda" <adem dot meda at gmail dot com> wrote in message
news:684606 at forums dot embarcadero dot com...

And I never agreed with UTF8String as a fundamental type. It is simply
another form of 1-bye string except that chars are byte-wise variable
length.

hehe, you mean unit-wise variable length, same as Unicode strings in windows
these days...?

You're, I believe, referring to surrogate pair stuff.

I never MS was pinnacle of long term planning.
david hoke

Posts: 616
Registered: 2/9/07
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 6:54 AM   in response to: Adem Meda in response to: Adem Meda
"Adem Meda" <adem dot meda at gmail dot com> wrote in message
news:684587 at forums dot embarcadero dot com...

But, we (the whole chain, that is) just didn't know what the future could
bring
up or didn't care enough.

So, would that ("didn't know the future") be why we started with 2-byte
Unicode, 4-byte unicode, etc, and somewhere in there wound up with utf8,
because someone figured out how to make it cover the whole range reasonably,
and generally compatibly, with existing 8-bit strings, being able to handle
both forward and backward searches (easily recognizing which items were
multi-byte and how many bytes)?

Or was it just because the person(s, Rob Pike and Ken Thompson) best suited
for the job was(ere)n't assigned the task initially? (Not really a
management level task I'd say...)

How much has that failure [of selecting appropriate (experienced-enough,
aged-enough, smart-enough, other?) personnel for the task] cost the entire
software industry, and by extension the technical world at large?
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 10:41 AM   in response to: david hoke in response to: david hoke
david hoke wrote:

"Adem Meda" <adem dot meda at gmail dot com> wrote in message
news:684587 at forums dot embarcadero dot com...

But, we (the whole chain, that is) just didn't know what the future could
bring up or didn't care enough.

So, would that ("didn't know the future") be why we started with 2-byte
Unicode, 4-byte unicode, etc, and somewhere in there wound up with utf8,
because someone figured out how to make it cover the whole range reasonably,
and generally compatibly, with existing 8-bit strings, being able to handle
both forward and backward searches (easily recognizing which items were
multi-byte and how many bytes)?

While UTF8 is a good compromise (at least doesn't have endianness headaches
build-it) when more than incompatibe system needs to communicate (transfer
information back and forth), it is, IMO, a good solution mostly nowadays used
at the wrong places.

UTF8 should not have been picked as an internal format. Not for applications,
not for whole OSes --it is a form of encoding for fox sake; and, why on erath
would you use encoded stuff in your own software as opposed to a native one
(such as fully 32bit 'char's)..

I have heard all the arguements why it was picked too.. "Otherwise, strings
would occupy too much memory" is the most used one. Except that I have never
seen a proof of it --i.e., we'd run out of memory if we used straight 32bit
codepoints.

Instead, at a time when CPU speeds have all but hit a ceiling, we've decided
(or, rather someone has decided for us) to have each character (codepoint) to
waste CPU cycles.

Needless to say, in the mean time, RAM has become abundant and dirt cheap; yet
we're stuck with UTF8 out our time-honored worry that strings might take too
much RAM..

Or was it just because the person(s, Rob Pike and Ken Thompson) best suited
for the job was(ere)n't assigned the task initially? (Not really a
management level task I'd say...)

I doubt anyone would have enough authroity to override the '8-bit'ters
shortsighted demands that world should revolve around them.

How much has that failure [of selecting appropriate (experienced-enough,
aged-enough, smart-enough, other?) personnel for the task] cost the entire
software industry, and by extension the technical world at large?

I have no idea what the amount is; but, every time I think about it, it
inflicts me enough mental pain that I could seriously damage whoever they were.
david hoke

Posts: 616
Registered: 2/9/07
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 10:58 AM   in response to: Adem Meda in response to: Adem Meda
"Adem Meda" <adem dot meda at gmail dot com> wrote in message
news:685057 at forums dot embarcadero dot com...

UTF8 should not have been picked as an internal format. Not for
applications,
not for whole OSes --it is a form of encoding for fox sake; and, why on
erath
would you use encoded stuff in your own software as opposed to a native
one
(such as fully 32bit 'char's)..

Hmm, and I thought, but haven't read/heard, that it was probably used for
*nix 'cuz it continues to work with all the standard library string
routines.

But maybe that was one of the less used arguments you heard.


I have heard all the arguements why it was picked too.. "Otherwise,
strings
would occupy too much memory" is the most used one. Except that I have
never
seen a proof of it --i.e., we'd run out of memory if we used straight
32bit
codepoints.

Instead, at a time when CPU speeds have all but hit a ceiling, we've
decided
(or, rather someone has decided for us) to have each character (codepoint)
to
waste CPU cycles.

Needless to say, in the mean time, RAM has become abundant and dirt cheap;
yet
we're stuck with UTF8 out our time-honored worry that strings might take
too
much RAM..

Or was it just because the person(s, Rob Pike and Ken Thompson) best
suited
for the job was(ere)n't assigned the task initially? (Not really a
management level task I'd say...)

I doubt anyone would have enough authroity to override the '8-bit'ters
shortsighted demands that world should revolve around them.

not sure I follow you there - Rob and Ken were supposedly responsible for
the 'working' form of utf8 we currently have...


How much has that failure [of selecting appropriate (experienced-enough,
aged-enough, smart-enough, other?) personnel for the task] cost the
entire
software industry, and by extension the technical world at large?

I have no idea what the amount is; but, every time I think about it, it
inflicts me enough mental pain that I could seriously damage whoever they
were.

And yet, you continue to promote the usage of non 8-bit strings - tsk tsk
tsk. Realizing that character size can only continue to grow, and will thus
continually require changes if one wants to have strings with a 'native'
(eh-oh) codepoint size, as hardware 'word' size continues to grow.
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 12:16 PM   in response to: david hoke in response to: david hoke
david hoke wrote:

"Adem Meda" <adem dot meda at gmail dot com> wrote in message
news:685057 at forums dot embarcadero dot com...

UTF8 should not have been picked as an internal format. Not for
applications, not for whole OSes --it is a form of encoding for fox sake;
and, why on erath would you use encoded stuff in your own software as
opposed to a native one (such as fully 32bit 'char's)..

Hmm, and I thought, but haven't read/heard, that it was probably used for
*nix 'cuz it continues to work with all the standard library string routines.

But maybe that was one of the less used arguments you heard.

I have heard that too. And, needless to say, I don't agree with that either.

It is amazing how easily we forget that the acronym 'UTF' stands for 'UCS
Transformation Format' where 'UCS' itself stands for 'Universal Character Set'.

So, if we expand it all, UTF stands for 'Universal Character Set Transformation
Format'.

I am not sure if you noticed the subtlety: It is NOT 'Universal Character Set'
but a 'Transformation Format'.

The fact that they claim it was easier to adopt "'cuz it continues to work with
all the standard library string routines" is less than half truth.

Full truth is that they were under pressure to do something fast, so they took
that 'easy' path.

They could have taken the proper path and overhauled each and every library
just like they had to when we moved from 32bit to 64bit but they didn't
because the 'easy' solution appeared to be good enough.

And, now we're in this stupid mess (where "mess" is my description).

Or was it just because the person(s, Rob Pike and Ken Thompson) best
suited for the job was(ere)n't assigned the task initially? (Not really a
management level task I'd say...)

I doubt anyone would have enough authroity to override the '8-bit'ters
shortsighted demands that world should revolve around them.

not sure I follow you there - Rob and Ken were supposedly responsible for the
'working' form of utf8 we currently have...

Finding a way (a universal one, a Rosetta Stone, if you like) that lets various
systems to talk to one another is an entirely different thing than finding a
proper native representation for codepages.

IOW, I am not denying the value of UTF8; it's just that (other people) decided
to use it internally as native representation.

That; the latter, was an utterly wrong decision, IMO.

And yet, you continue to promote the usage of non 8-bit strings - tsk tsk
tsk. Realizing that character size can only continue to grow, and will thus
continually require changes if one wants to have strings with a 'native'
(eh-oh) codepoint size, as hardware 'word' size continues to grow.

Yes, 'character' sizes are likely to grow (see below as an 'insert' on the
subject), but it doesn't happen often, and when it does, all you need to do a
one-time overhaul. IOW, a few days/weeks of brainpower needs to be expended; as
opposed to wasting extra CPU-cycles everytime you want to access a 'character'
in a string.

<insert>
While it is likely that, for a long time, we will not need more codepages than
can be covered by 24-bit (3-byte), there is the issue of casing (and others)
that actually require each codepage/char (in a string) to be associated with a
language.

IOW, if we have to do it right, we have to know (have information about) what
language each character in a string belongs to.

Since there are more than 256 langugaes (dead or alive) we will need another
2-bytes (16-bit) for each character.

And, that takes us to 5-byte (40-bit) codepages.

I am sure Unicode people are aware of this, but it's time hasn't come yet (I
can just about hear the uproar from the '8-bit'ter crowd.

So as an interim (palliative, more likely) measure, they are busy publishing
LowerCase/UpperCase/TitleCase tables while ignoring embedded language
association.
</insert>
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 12:57 PM   in response to: Adem Meda in response to: Adem Meda
Am 05.07.2014 02:57, schrieb Adem Meda:
Dalija Prasnikar wrote:

If simpler code can do the job and has been doing job for ages on even less
powerfull computers than we have today, how can I explain that I need to
spend valuable time modifying working code just because my development tool
got crappier.

Could it be because you started off wrong? I mean, we all abused strings as
in-RAM data storage but to claim that it is a right to continue to do so is,
IMO, strecthing it a bit too far.

Hello,

think twice: some scientific fields do have alphabets consisting of only
A, C, G, T and U. Fits perfectly in one byte per char and long strings
of those do exist in those fields and are perfect reasonst for using
strings to store them, as they often do operations like pos on them.

Now think again how much it speeds up an algorithm where you have to
search repeatedly for various things in the same string when you can
hold the whole string in memory for the complete time.


I don't need to interpret any characters above 127, and I don't need to show
any data during processing.

There, then, you already have the opportunity to reduce the memory footprint of
your 200 MB XML string by (probably more than) 50 % if only you read it into
memory as 2 chars per byte.

In short: At least for the example you gave, I can't find a reason to
sympathise with your complaints when you're reading the whole file into RAM in
one go which you shouldn't; and, use 'string' for it when you really should use
a more suitable structure. What's worse is that implementing any of this is not
at all complicated.

The above cited example makes memory mapped files look 2nd choice only.
Sorry.

Clkeaning up is nice now and then, but there was no need to do away with
ALL 8-bit string types.

Greetings

Markus
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 3:56 PM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

think twice: some scientific fields do have alphabets consisting of only
A, C, G, T and U. Fits perfectly in one byte per char and long strings
of those do exist in those fields and are perfect reasonst for using
strings to store them, as they often do operations like pos on them.

I simply do not agree that it is a good idea to read such data into a string in
memory.

Not only does it waste RAM, it would also be slow.

But, if I had to: I'd load all that DNA data into RAM (if I have to) as a blob
and then typecast that blob as a 'PChar' to do my 'text operations' on them.

Ah, another thing: The fact that those enzymes are represented by a single char
each, their combinations (a soup of chars) does not make a text that ordinary
humans can comprehend.

The above cited example makes memory mapped files look 2nd choice only.
Sorry.

I never said anything about memory mapping a file on a HDD. You have a few more
choices if speed is the overriding concern. You can use faster SSDs (RAIDed)
or, better still, use RAM Disks; or use the suggestion I made in the first
paragraph above.

In short, I am not against fast solutions; on the contrary, I am all for them.

But, the data type 'string' (and its realtives) is rarely a good way to go
about it.

Cleaning up is nice now and then, but there was no need to do away with
ALL 8-bit string types.

Except for a handful of languages, 8-bit string type is not usable.

Pretending it to be otherwise is either abusing strings (to contain binary
data) or ignroing the rest of the world.
david hoke

Posts: 616
Registered: 2/9/07
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 7:11 AM   in response to: Adem Meda in response to: Adem Meda
"Adem Meda" <adem dot meda at gmail dot com> wrote in message
news:684667 at forums dot embarcadero dot com...

Except for a handful of languages, 8-bit string type is not usable.

Pretending it to be otherwise is either abusing strings (to contain binary
data) or ignroing the rest of the world.

So apparently you also are among those who do not realize that 'strings' are
binary data, too.
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 10:41 AM   in response to: david hoke in response to: david hoke
david hoke wrote:

"Adem Meda" <adem dot meda at gmail dot com> wrote in message
news:684667 at forums dot embarcadero dot com...

Except for a handful of languages, 8-bit string type is not usable.

Pretending it to be otherwise is either abusing strings (to contain binary
data) or ignroing the rest of the world.

So apparently you also are among those who do not realize that 'strings' are
binary data, too.

Yeah, thank you for letting me in on the secret after all these years ;)
Robert Dawson

Posts: 211
Registered: 7/28/00
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 7:19 PM   in response to: Markus Humm in response to: Markus Humm
"Markus Humm" wrote

think twice: some scientific fields do have alphabets consisting of only
A, C, G, T and U. Fits perfectly in one byte per char and long strings
of those do exist in those fields and are perfect reasonst for using
strings to store them, as they often do operations like pos on them.

Not entirely. See http://www.bioinformatics.org/sms2/iupac.html

While one letter per base can suffice in simple cases, additional letters
are often used, and even appear in the NCBI Reference Sequence Database
(http://www.ncbi.nlm.nih.gov/refseq/ ). where you can run into things like
...(A/C)... where (A/C) means the base at that position is either A or C (=
iupac M)

Once you start caring about different bond chemistry between the bases, or
different base types, single-letter notation becomes pretty much impossible.
The following is from
https://www.idtdna.com/analyzer/Applications/OligoAnalyzer/

Base Notations
DNA = A, C, G, T and U, I
Mixed Bases = Please enter bases in UPPERCASE
Phosphorothioated DNA = A*, G*, C*, T*
RNA = rA, rG, rC, rU
Phosphorothioated RNA = rA*, rG*, rC*, rU*
2'O-Methyl RNA = mA, mG, mC, mU
Phosphorothioated 2'O-Methyl RNA = mA*, mG*, mC*, mU*
Locked Nucleic Acid (LNA) = +A, +G, +C, +T

Bottom line: naively parsing a DNA sequence with pos() is actually quite
error prone unless one can use a prefilter.

bobD
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 2:23 AM   in response to: Robert Dawson in response to: Robert Dawson
Hello,

ok, learned something new now.

Greetings

Markus
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 8:28 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Adem Meda wrote:
Dalija Prasnikar wrote:

Adem Meda wrote:
Even though 500 MB is peanuts for my hardware, let's (for the
sake of discussion) consider it too big to load into RAM: Why
would I load it into RAM and why not use memory mapped files?

It is not peanuts on 32-bit Windows.

Good thing, then, that memory-mapping is also available in 32-bit
Windows; if only people concerned about memory would use it ;)

If simpler code can do the job and has been doing job for ages on
even less powerfull computers than we have today

Then how come you run ínto memory problems all of a sudden? Apparently
the design is not scalable. And "until now, it has not failed us" is
not a valid argument, sorry.
--
Rudy Velthuis (TeamB) http://www.teamb.com

"Facts are the enemy of truth."
-- Don Quixote - "Man of La Mancha"
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 5:37 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
Dalija Prasnikar wrote:

Adem Meda wrote:
Dalija Prasnikar wrote:

Adem Meda wrote:
Even though 500 MB is peanuts for my hardware, let's (for the
sake of discussion) consider it too big to load into RAM: Why
would I load it into RAM and why not use memory mapped files?

It is not peanuts on 32-bit Windows.

Good thing, then, that memory-mapping is also available in 32-bit
Windows; if only people concerned about memory would use it ;)

If simpler code can do the job and has been doing job for ages on
even less powerfull computers than we have today

Then how come you run ínto memory problems all of a sudden? Apparently
the design is not scalable. And "until now, it has not failed us" is
not a valid argument, sorry.

I didn't run into memory problem, I would run into memory problem if I
would change my library to use unicode strings as suggested.

Dalija Prasnikar
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 11:56 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Rudy Velthuis (TeamB) wrote:
Dalija Prasnikar wrote:

Adem Meda wrote:
Dalija Prasnikar wrote:

Adem Meda wrote:
Even though 500 MB is peanuts for my hardware, let's (for
the sake of discussion) consider it too big to load into
RAM: Why would I load it into RAM and why not use memory
mapped files?

It is not peanuts on 32-bit Windows.

Good thing, then, that memory-mapping is also available in
32-bit Windows; if only people concerned about memory would use
it ;)

If simpler code can do the job and has been doing job for ages on
even less powerfull computers than we have today

Then how come you run ínto memory problems all of a sudden?
Apparently the design is not scalable. And "until now, it has not
failed us" is not a valid argument, sorry.

I didn't run into memory problem, I would run into memory problem if
I would change my library to use unicode strings as suggested.

Then ISTM that something is wrong with such design. If such a change
can already cause problems, the design should be reconsidered.
--
Rudy Velthuis (TeamB) http://www.teamb.com

"We have a political system that awards office to the most
ruthless, cunning, and selfish of mortals, then act surprised
when those willing to do anything to win power are equally
willing to do anything with it."
-- Michael Rivero
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 12:09 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
Dalija Prasnikar wrote:

I didn't run into memory problem, I would run into memory problem if
I would change my library to use unicode strings as suggested.

Then ISTM that something is wrong with such design. If such a change
can already cause problems, the design should be reconsidered.

What I am trying to show is that doubling size of character is not a small
change. It can drastically change memory requirements for application.

Dalija Prasnikar
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 1:16 PM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Rudy Velthuis (TeamB) wrote:
Dalija Prasnikar wrote:

I didn't run into memory problem, I would run into memory problem
if I would change my library to use unicode strings as suggested.

Then ISTM that something is wrong with such design. If such a change
can already cause problems, the design should be reconsidered.

What I am trying to show is that doubling size of character is not a
small change. It can drastically change memory requirements for
application.

Only for some "brute force" applications that load (too) many strings
into memory at once, like apparently yours. Most applications will not
notice a big difference.
--
Rudy Velthuis (TeamB) http://www.teamb.com

"A conservative is a man who is too cowardly to fight and too
fat to run."
-- Elbert Hubbard
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 8:26 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Adem Meda wrote:
Dalija Prasnikar wrote:

Adem Meda wrote:
Dalija Prasnikar wrote:

What do you convert that UTF8 text/data to and from?

I don't convert it, I leave it in UTF8 format and do direct
processing. If I would use Delphi unicode string type I
would had two unneccessary conversions that take time +
doubled memory requirements (that would mean that processing
is no longer possible on 32-bit Windows I still have to
support).

(Disclaimer: Even though I work with a lot of text data, I
don't do mobile.)

I don't know what sort of applications you deal with that
doubling (or even quadrubling) the memory used by text data
means prohibitively high usage of RAM, or makes the app too
slow.

Try loading 500MB XML in memory and do something with it. With
UTF8 it will consume 500MB and another 500MB for changes, double
that and you are out of memory real soon.

Even though 500 MB is peanuts for my hardware, let's (for the sake
of discussion) consider it too big to load into RAM: Why would I
load it into RAM and why not use memory mapped files?

It is not peanuts on 32-bit Windows. And strings make that code
simple, clean and easy to understand and change. Anything else would
make that code more complex.

Why?

--
Rudy Velthuis (TeamB) http://www.teamb.com

"While the word is yet unspoken, you are master of it; when
once it is spoken, it is master of you."
-- Arab proverb
Matthew Jones

Posts: 337
Registered: 1/25/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 4:14 AM   in response to: Adem Meda in response to: Adem Meda
Adem Meda wrote:

Even though 500 MB is peanuts for my hardware

The iPhone 4S had 512MB RAM in total. The 5 and 5S have 1GB.

Why would I load it into RAM
and why not use memory mapped files?

The point here is that if you knew that it would ever get that big
you'd not do it that way in the first place, but you start off with
small data and then users get to have a say. This same transfer
application was designed (by someone else) to handle about ten
connections. Then one day the customer came to me and said it was too
slow when it got to a thousand connections, and they needed to upgrade
it to two thousand.

Today, you'd be mad to thing that 500Mb on an iPhone is a lot of data.
But then 20 seconds of iPhone movie is 37.7Mb, and soon you are getting
large data.

You don't want to be converting stuff to another format for the sake of
it.

Matthew
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 10:41 AM   in response to: Matthew Jones in response to: Matthew Jones
Matthew Jones wrote:

Adem Meda wrote:

Even though 500 MB is peanuts for my hardware

The iPhone 4S had 512MB RAM in total. The 5 and 5S have 1GB.

There you are. If you need to load 200 MB XML text into RAM, Apple shouldn't be
among your target apps.

Or, alternatively, you need to use techniques to circumvent that problem: i.e.
don't read it all into memory, use techniques that we had to use when personal
computers had similar constraints.

IOW, You don't have to invent anything new, it is all there in everyone's near
past.

Why would I load it into RAM
and why not use memory mapped files?

The point here is that if you knew that it would ever get that big
you'd not do it that way in the first place, but you start off with
small data and then users get to have a say. This same transfer
application was designed (by someone else) to handle about ten
connections. Then one day the customer came to me and said it was too
slow when it got to a thousand connections, and they needed to upgrade
it to two thousand.

I can understand your position especially when the customer cannot (or will
not) comprehend the reasons why his/her demand is not meaningful.

But, in all honesty, I'd refuse to bow to such demands --it is like wanting to
use a family car as an earth moving machinery [there's a car analogy for you,
enriched with trucks :) ].

Today, you'd be mad to think that 500Mb on an iPhone is a lot of data.
But then 20 seconds of iPhone movie is 37.7Mb, and soon you are getting
large data.

Well.. no one (using tablets or smartphones) loads the full movie into RAM,
they use portions of it as necessary.

And, those that need to access 200 MB files should, IMO, either save it to disk
and then read portions as necessary, or also write a server app that does the
grunt work on a 'proper' hardware.

You don't want to be converting stuff to another format for the sake of
it.

Are you referring to the use of UTF8? ;)
Matthew Jones

Posts: 337
Registered: 1/25/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 8, 2014 1:32 AM   in response to: Adem Meda in response to: Adem Meda
Adem Meda wrote:

Matthew Jones wrote:

Adem Meda wrote:

Even though 500 MB is peanuts for my hardware

The iPhone 4S had 512MB RAM in total. The 5 and 5S have 1GB.

There you are. If you need to load 200 MB XML text into RAM, Apple
shouldn't be among your target apps.

Well, I'd agree. The key here is not what is best or right, or whether
users are sensible. The key is that you might have an XML file to load
from a service that is in UTF8. When you load it into your limited RAM
device, it doubles in size because the tool you are using doesn't have
the UTF8String it used to have. Okay, no big problem really, but you
were rather hoping that you could use the analysis library that you
have working and reliable on your desktop solution. Unfortunately, that
can't be used because it uses the UTF8String that is the native data
format. Okay, no big deal, you just write a new solution, but when you
are having to start over you might just as well find a solution that is
a better fit. As it happens, that's what I did. Delphi is now my
back-end server, and passes UTF8 strings to the clients (half the data
on the wire...)

One thing that amazes me is how no one ever accepts that other people's
ideas about product changes are allowed. Instead they have to be
refuted, hence we get these long threads that go nowhere. If it were
from Embarcadero developers I could understand perhaps, but half the
negativity is coming from the camp which is opposite to the "sky is
falling" but "the sky is perfect". Just an observation...

Matthew
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 8, 2014 4:09 AM   in response to: Matthew Jones in response to: Matthew Jones
Matthew Jones wrote:

As it happens, that's what I did. Delphi is now my
back-end server, and passes UTF8 strings to the clients (half the data
on the wire...)

It seems quite a lot of professional coders have adopted the strategy they
complain about others: i.e. if it compiles, ship it.

Here's what I mean by that: If I needed to pass a large file to a mobile
device, I'd make sure there's a server that handles the grunt work.

But, that's not enough, I'd also design the server so that it supplies only the
relevant portion/segment of the file on demand (and, if necessary and/or
appropriate, in compressed form to save bandwidth charges).

We all know that adding these features isn't rocket science; except that,
naturally, getting everything working properly can take sometime time.

I too am all for rapid solutions, but not spending the time and effort to make
it a proper solution, looks suspiciously like the priority and focus are on
quick bucks.
Matthew Jones

Posts: 337
Registered: 1/25/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 8, 2014 6:12 AM   in response to: Adem Meda in response to: Adem Meda
Adem Meda wrote:

But, that's not enough, I'd also design the server so that it
supplies only the relevant portion/segment of the file on demand
(and, if necessary and/or appropriate, in compressed form to save
bandwidth charges).

Sure. I think the key is that you can't claim code re-use, as
Embarcadero do, when you can't err, re-use the code you have. Are these
big issues? No, but they are issues and they certainly stopped me using
Delphi as a mobile solution. I'd used the first release just fine for
proof of concept, and then while saying it was code compatible, it
wasn't. And then you get the usual suspects just saying we are being
precious or similar.

Real world development is not simple and not just programming.

Matthew
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 8, 2014 7:43 AM   in response to: Matthew Jones in response to: Matthew Jones
Matthew Jones wrote:

Adem Meda wrote:

But, that's not enough, I'd also design the server so that it
supplies only the relevant portion/segment of the file on demand
(and, if necessary and/or appropriate, in compressed form to save
bandwidth charges).

Sure. I think the key is that you can't claim code re-use, as
Embarcadero do, when you can't err, re-use the code you have.

Hmm.. I see your point.

Frankly, I am not at all sure that Delphi will become multi-platform given the
amount of baggage it already carries.

I have a feeling if EMB can ever pull it off, it will more likely be a 'Delphi
for multi-platform' but will leave a lot of people with 'legacy' code behind.

Are these big issues? No, but they are issues and they certainly
stopped me using Delphi as a mobile solution. I'd used the first
release just fine for proof of concept, and then while saying it
was code compatible, it wasn't. And then you get the usual
suspects just saying we are being precious or similar.

I hear some favorable noises about Basic4android. Basic might turn out to take
its own back, after all.

Real world development is not simple and not just programming.

Nothing properly done is.
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 15, 2014 1:01 PM   in response to: Adem Meda in response to: Adem Meda
Am 08.07.2014 13:09, schrieb Adem Meda:
Matthew Jones wrote:

As it happens, that's what I did. Delphi is now my
back-end server, and passes UTF8 strings to the clients (half the data
on the wire...)

It seems quite a lot of professional coders have adopted the strategy they
complain about others: i.e. if it compiles, ship it.

Here's what I mean by that: If I needed to pass a large file to a mobile
device, I'd make sure there's a server that handles the grunt work.

Hello,

that's in most cases a useful solution, but it won't cover all possible
scenarious. If there is no server access at the places where the
equipment shall be operated you're out of luck with that kind of solution!

Greetings

Markus
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 15, 2014 3:10 PM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

If there is no server access at the places where the equipment
shall be operated you're out of luck with that kind of solution!

That would definitely be the case.

But, if I came across to someone with those constraints, my immediate
response would be something like "call back in 5+ years when we
will have mobile HW with at least 32 GB RAM and TBs of storage".
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 16, 2014 1:17 PM   in response to: Adem Meda in response to: Adem Meda
Am 16.07.2014 00:10, schrieb Adem Meda:
Markus Humm wrote:

If there is no server access at the places where the equipment
shall be operated you're out of luck with that kind of solution!

That would definitely be the case.

But, if I came across to someone with those constraints, my immediate
response would be something like "call back in 5+ years when we
will have mobile HW with at least 32 GB RAM and TBs of storage".

WHat a pity that plroblems which somebody wants to get solved TODAY
won't wait for another 5 years or so (at least most problems of that
sort). :-(

Greetings

Markus
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 17, 2014 3:40 PM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

But, if I came across to someone with those constraints, my immediate
response would be something like "call back in 5+ years when we
will have mobile HW with at least 32 GB RAM and TBs of storage".

WHat a pity that plroblems which somebody wants to get solved TODAY
won't wait for another 5 years or so (at least most problems of that
sort). :-(

Hah! 5 years is nothing.

I know of so many problems that have been waiting for solutions for decades if
not centuries.

Where is my long-promised flying car?
Bruce McGee

Posts: 1,716
Registered: 9/30/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 3:53 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Try loading 500MB XML in memory and do something with it. With UTF8
it will consume 500MB and another 500MB for changes, double that and
you are out of memory real soon.

Loading a 500MB XML file into a DOM on a mobile device sounds like a
design fail.

In fact, I'd hesitate to do this in a desktop application. More for
performance reasons than memory concerns, but still...

--
Regards,
Bruce McGee
Glooscap Software
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 4:12 AM   in response to: Bruce McGee in response to: Bruce McGee
Bruce McGee wrote:
Dalija Prasnikar wrote:

Try loading 500MB XML in memory and do something with it. With UTF8
it will consume 500MB and another 500MB for changes, double that and
you are out of memory real soon.

Loading a 500MB XML file into a DOM on a mobile device sounds like a
design fail.

I never said I need to load such file on mobile, even though I can see that
more and more tablets will be used for heavier business use.

In fact, I'd hesitate to do this in a desktop application. More for
performance reasons than memory concerns, but still...

I am not loading into DOM at all. Just plain UTF8String and nothing more.

Once again, my needs for mobile are not that complex, but I need to reuse
existing library that cannot change due to desktop requirements.

But I can also see beyond my needs and it is very likely that others
may need compact string types on mobile because of their mobile requirements.

Dalija Prasnikar
Bruce McGee

Posts: 1,716
Registered: 9/30/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 4:33 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Bruce McGee wrote:
Dalija Prasnikar wrote:

Try loading 500MB XML in memory and do something with it. With
UTF8 it will consume 500MB and another 500MB for changes, double
that and you are out of memory real soon.

Loading a 500MB XML file into a DOM on a mobile device sounds like a
design fail.

I never said I need to load such file on mobile, even though I can
see that more and more tablets will be used for heavier business use.

In fact, I'd hesitate to do this in a desktop application. More for
performance reasons than memory concerns, but still...

I am not loading into DOM at all. Just plain UTF8String and nothing
more.

Once again, my needs for mobile are not that complex, but I need to
reuse existing library that cannot change due to desktop requirements.

But I can also see beyond my needs and it is very likely that others
may need compact string types on mobile because of their mobile
requirements.

Dalija Prasnikar


--
Regards,
Bruce McGee
Glooscap Software

Bruce McGee

Posts: 1,716
Registered: 9/30/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 4:47 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Bruce McGee wrote:
Dalija Prasnikar wrote:

Try loading 500MB XML in memory and do something with it. With
UTF8 it will consume 500MB and another 500MB for changes, double
that and you are out of memory real soon.

Loading a 500MB XML file into a DOM on a mobile device sounds like a
design fail.

I never said I need to load such file on mobile, even though I can
see that more and more tablets will be used for heavier business use.

My mistake. Thanks for clarifying.

In fact, I'd hesitate to do this in a desktop application. More for
performance reasons than memory concerns, but still...

I am not loading into DOM at all. Just plain UTF8String and nothing
more.

Good to know.

Once again, my needs for mobile are not that complex, but I need to
reuse existing library that cannot change due to desktop requirements.

I have sympathy for that position. I'm huge on code compatibility.

But I don't think enough people think it's a big enough issue that
Embarcadero will change their plans.

But I can also see beyond my needs and it is very likely that others
may need compact string types on mobile because of their mobile
requirements.

I honestly don't think it's going to be that big a deal for most people.

--
Regards,
Bruce McGee
Glooscap Software
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 5:15 AM   in response to: Bruce McGee in response to: Bruce McGee
Bruce McGee wrote:

But I don't think enough people think it's a big enough issue that
Embarcadero will change their plans.


If top voted QC report doesn't show that it is issue for quite number
of people than I don't know what else to say. Maybe people voted
for it just because I am such likable person.


But I can also see beyond my needs and it is very likely that others
may need compact string types on mobile because of their mobile
requirements.

I honestly don't think it's going to be that big a deal for most people.

I don't think that all Delphi users need mobile support at the moment, some
might need it later on and some will not be happy with state of things.

Dalija Prasnikar
Bruce McGee

Posts: 1,716
Registered: 9/30/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 5:30 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Bruce McGee wrote:

But I don't think enough people think it's a big enough issue that
Embarcadero will change their plans.


If top voted QC report doesn't show that it is issue for quite number
of people than I don't know what else to say. Maybe people voted
for it just because I am such likable person.

That's not the only consideration, but no, apparently that isn't
enough. btw, I also voted for it.

As I wrote before, I think it would take direct appeals with more
concrete details from enough people who are directly affected by the
decision.

A few people being legitimately put out won't make them change their
minds no matter how many times those few people repeat their arguments.

If that worked, I'd have desktop Linux support by now.

I don't think that all Delphi users need mobile support at the
moment, some might need it later on and some will not be happy with
state of things.

I don't think it's going to be a big deal for most mobile developers,
either.

--
Regards,
Bruce McGee
Glooscap Software
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 10:36 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija,

| I don't think that all Delphi users need mobile support at the
| moment, some might need it later on and some will not be happy with
| state of things.

I could use it now. And I am NOT "happy with the state of things!"

As a result I have had to cancel all plans for "going mobile." <sigh>

--

Q

07/04/2014 10:35:05

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 1:14 AM   in response to: Quentin Correll in response to: Quentin Correll
Quentin Correll wrote:
Dalija,

| I don't think that all Delphi users need mobile support at the
| moment, some might need it later on and some will not be happy with
| state of things.

I could use it now. And I am NOT "happy with the state of things!"

As a result I have had to cancel all plans for "going mobile." <sigh>

I fully understand your position.

Dalija Prasnikar
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 10:03 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija,

| | As a result I have had to cancel all plans for "going mobile."
| | <sigh>

| I fully understand your position.

I've been in "this business" fifty-nine years. And this is the first
time I can remember that an "advance" (Yeah, right.) in technology
makes it MORE difficult to do things that have been being done for
decades!!! The "young minds" have royally f****** things up!!!!!!!

--

Q

07/05/2014 10:00:06

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Konstantine Pou...

Posts: 128
Registered: 11/3/06
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 10, 2014 3:35 PM   in response to: Quentin Correll in response to: Quentin Correll
I've been in "this business" fifty-nine years. And this is the first
time I can remember that an "advance" (Yeah, right.) in technology
makes it MORE difficult to do things that have been being done for
decades!!! The "young minds" have royally f****** things up!!!!!!!

While present to varied degree in other industries the whole
software industry is undisputed champion in this department.
Bunch of primadonnas thinking that the whole world should
bend over to their vision or else ...
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 10, 2014 3:57 PM   in response to: Konstantine Pou... in response to: Konstantine Pou...
Konstantine,

| While present to varied degree in other industries the whole
| software industry is undisputed champion in this department.
| Bunch of primadonnas thinking that the whole world should
| bend over to their vision or else ...

Agreed.

--

Q

07/10/2014 15:57:16

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Mike Margerum

Posts: 590
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 1:28 PM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Try loading 500MB XML in memory and do something with it. With UTF8 it will
consume 500MB and another 500MB for changes, double that and you are out
of memory real soon.

Dalija Prasnikar
IF i were parsing a 500MB xml file i darn sure wouldn't be using a Tree
based parser in any language or string format. Its easy enough to build
an event driven parser.
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 8:15 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Adem Meda wrote:
Dalija Prasnikar wrote:

What do you convert that UTF8 text/data to and from?

I don't convert it, I leave it in UTF8 format and do direct
processing. If I would use Delphi unicode string type I would had
two unneccessary conversions that take time + doubled memory
requirements (that would mean that processing is no longer
possible on 32-bit Windows I still have to support).

(Disclaimer: Even though I work with a lot of text data, I don't do
mobile.)

I don't know what sort of applications you deal with that doubling
(or even quadrubling) the memory used by text data means
prohibitively high usage of RAM, or makes the app too slow.

Try loading 500MB XML in memory and do something with it.

That is something I would never do, even if I had to process that much.
ISTM that loading all of this in memory at once is not very scalable,
and I'd guess I'd find other ways to do this.

But if that is the kind of code you'd want to port to mobile with much
more limited memory, you would have to redesign it anyway, even if you
could use UTF8String. If that is not the kind of code, then I don't
quite understand the problem, since then the string element size should
not matter much.
--
Rudy Velthuis (TeamB) http://www.teamb.com

"Any fool can make a rule, and any fool will mind it."
-- Henry David Thoreau
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 10:12 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy,

| That is something I would never do, even if I had to process that
| much. ISTM that loading all of this in memory at once is not very
| scalable, and I'd guess I'd find other ways to do this.

It's not about "other ways," it's about performance.

I did a test with my big app. What takes a couple of hours to run with
very large DynArrays in ram took over nine hours using NexusDB, the
fastest DBMS I know of. A TOTALLY UNACCEPTABLE run-time. <sigh>

| But if that is the kind of code you'd want to port to mobile with much
| more limited memory, you would have to redesign it anyway, even if you
| could use UTF8String. If that is not the kind of code, then I don't
| quite understand the problem, since then the string element size
| should not matter much.

There is NO "re-design" possible for my app due the math involved. But
if the "mobile" device had the same cpu-ram capability of my desktop PC
it would be a snap!!!

--

Q

07/05/2014 10:03:59

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 11:44 AM   in response to: Quentin Correll in response to: Quentin Correll
Quentin Correll wrote:

Rudy,

That is something I would never do, even if I had to process that
much. ISTM that loading all of this in memory at once is not very
scalable, and I'd guess I'd find other ways to do this.

It's not about "other ways," it's about performance.

I am not sure if I would sacrifice scalability for performance. I would
localize and split my data into portions that can be processed at full
performance, and only load new portions when that is necessary. That
should hardly affect performance, if at all. With memory mapped files,
you have the advantage that you can let the OS do the mapping and
caching etc.

I did a test with my big app. What takes a couple of hours to run
with very large DynArrays in ram took over nine hours using NexusDB,
the fastest DBMS I know of. A TOTALLY UNACCEPTABLE run-time. <sigh>

Who is talking about using a DBMS instead of arrays? I am certainly
not. I am talking about not loading all data into memory at once, but
using mekory intelligently.

But if that is the kind of code you'd want to port to mobile with
much more limited memory, you would have to redesign it anyway,
even if you could use UTF8String. If that is not the kind of code,
then I don't quite understand the problem, since then the string
element size should not matter much.

There is NO "re-design" possible for my app due the math involved.

I meant Dalijah. I doubt your app with very large dynarrays in memory
could run on a current mobile platform at all, no matter which language
you would choose.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"This country, with its institutions, belongs to the people who
inhabit it. Whenever they shall grow weary of the existing
government, they can exercise their constitutional right of
amending it, or exercise their revolutionary right to overthrow
it."
-- Abraham Lincoln

Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 2:39 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy,

| Who is talking about using a DBMS instead of arrays? I am certainly
| not. I am talking about not loading all data into memory at once, but
| using mekory intelligently.

Yeah, I know. It was just that is what my big app requires and all I
had to use for testing. <g>

| I meant Dalijah. I doubt your app with very large dynarrays in memory
| could run on a current mobile platform at all, no matter which
| language you would choose.

LOL! Yep! Very true!

And I wasn't smart enough to figure that out until after I had spent a
couple of grand USD on Apple-crap. I was blinded by the neat idea of
how my app's functionality could be used on a portable-mobile device.
Some tools just aren't meant to handle some tasks. I didn't give the
issues anywhere near enough serious thought until AFTER I tried jumping
on the band-wagon and was promptly thrown off with a resounding thump.
Mea culpa. <g>

I now just have to treat my poorly analyzed experience as an expensive
"learning experience." ;-)

Q

07/06/2014 14:23:48

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 8, 2014 4:37 PM   in response to: Quentin Correll in response to: Quentin Correll
Quentin Correll wrote:

And I wasn't smart enough to figure that out until after I had spent a
couple of grand USD on Apple-crap.

You can think of Apple what you like, but they don't sell crap.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"All civilization has from time to time become a thin crust
over a volcano of revolution."
-- Havelock Ellis
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 8, 2014 4:49 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy,

| You can think of Apple what you like, but they don't sell crap.

It was to me. <g>

--

Q

07/08/2014 16:49:08

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Konstantine Pou...

Posts: 128
Registered: 11/3/06
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 10, 2014 3:39 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
You can think of Apple what you like, but they don't sell crap.

For some people it is invaluable to others it is crap.
As it stands now for me personally the only reason I would
ever own their product is if I have to offer my product on
their platform.
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 15, 2014 1:04 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 09.07.2014 01:37, schrieb Rudy Velthuis (TeamB):
Quentin Correll wrote:

And I wasn't smart enough to figure that out until after I had spent a
couple of grand USD on Apple-crap.

You can think of Apple what you like, but they don't sell crap.

Hello,

not that other vendors aren't doing the same: use of non replacible
rechargeable batteries. But such a design is eco-crap!
(if favours design over eco friendlines and that idea is crap)

In the same direction goes soldered RAM in laptops just to get them 2mm
or so thinner and requiring specialized shops for RAM upgrade (if
possible at all).

Greetings

Markus
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 15, 2014 3:07 PM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

Am 09.07.2014 01:37, schrieb Rudy Velthuis (TeamB):
Quentin Correll wrote:

And I wasn't smart enough to figure that out until after I had
spent a >> couple of grand USD on Apple-crap.

You can think of Apple what you like, but they don't sell crap.

not that other vendors aren't doing the same: use of non replacible
rechargeable batteries.

Not sure what you are saying. Apple doesn't sell crap. I have no idea
how your objection matters.
--
Rudy Velthuis (TeamB) http://www.teamb.com

"Simplicity is the ultimate sophistication."
-- Leonardo da Vinci
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 16, 2014 1:19 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 16.07.2014 00:07, schrieb Rudy Velthuis (TeamB):
Markus Humm wrote:

Am 09.07.2014 01:37, schrieb Rudy Velthuis (TeamB):
Quentin Correll wrote:

And I wasn't smart enough to figure that out until after I had
spent a >> couple of grand USD on Apple-crap.

You can think of Apple what you like, but they don't sell crap.

not that other vendors aren't doing the same: use of non replacible
rechargeable batteries.

Not sure what you are saying. Apple doesn't sell crap. I have no idea
how your objection matters.

Hello,

selling electronic products where the part which wears out quickes
(means battery) cannot be properly replaced (without dismounting half of
the product and most likely damaging it while doing this) is crap in
today's times.

Products should get more eco friendly and not less eco friendly.

Now which iOS product from Apple has replacible batteries for isntance?
And afaik a few of their laptops don't have them any more as well (there
I might be wrong though).

Greetings

Markus
Matthew Jones

Posts: 337
Registered: 1/25/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 17, 2014 8:17 AM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

selling electronic products where the part which wears out quickes
(means battery) cannot be properly replaced (without dismounting half
of the product and most likely damaging it while doing this) is crap
in today's times.

Products should get more eco friendly and not less eco friendly.

Yes, but it isn't true about Apple products. If you put your prejudice
away a minute, you'll see on the internet that there are plenty of
people who can replace your battery in an iPhone without much hassle.
The iPad has a special rig in store that allows them to be opened
without hassle. The benefit of a solid phone is that they are more
robust, and less likely to suffer damage from dropping, which means
they should last longer, and that's good for the environment. The
battery is good for years anyway - my son uses my old iPhone 4 and gets
a full day of too much use from it.

Matthew
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 17, 2014 1:26 PM   in response to: Matthew Jones in response to: Matthew Jones
Am 17.07.2014 17:17, schrieb Matthew Jones:
Markus Humm wrote:

selling electronic products where the part which wears out quickes
(means battery) cannot be properly replaced (without dismounting half
of the product and most likely damaging it while doing this) is crap
in today's times.

Products should get more eco friendly and not less eco friendly.

Yes, but it isn't true about Apple products. If you put your prejudice
away a minute, you'll see on the internet that there are plenty of
people who can replace your battery in an iPhone without much hassle.
The iPad has a special rig in store that allows them to be opened
without hassle. The benefit of a solid phone is that they are more
robust, and less likely to suffer damage from dropping, which means
they should last longer, and that's good for the environment. The
battery is good for years anyway - my son uses my old iPhone 4 and gets
a full day of too much use from it.

Matthew

Hello,

TV channel once made a test with an iPhone and a Samsung S3 or S4. Both
dropped accidentially into some glass of beer at a bar.

The Samsung's battery could be taken out without any tools etc.
(which you normally don't have at hands when such an accident happens)
After 3 days or so of drying the Samsung device worked flawlessly again,
but the iPhone was half broken in functionality as the water in the
device caused electric shortages damaging components.

And why should I need some shop replacing the battery for me? As can be
seen in my example designs are possible where the end user can do it
without much hassle himself thus not being bound to any service times of
any store etc.

Greetings

Markus
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 17, 2014 4:23 PM   in response to: Markus Humm in response to: Markus Humm
The Samsung's battery could be taken out without any tools etc.

While I agree with everything (and more) you say about Apple, Samsung isn't
totally without sin: I have a Samsung tablet (specifically 10.1) and it simply
demands that the charger be made by Samsung.

Latter models (12.2, for example) does not seem to have that, but still. It was
a very Apple move by Samsung.
Matthew Jones

Posts: 337
Registered: 1/25/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 17, 2014 11:56 PM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

TV channel once made a test with an iPhone and a Samsung S3 or S4.
Both dropped accidentially into some glass of beer at a bar.

Well, my years in computer repairs tells me that the iPhone user got
the best deal. That Samsung will last a few more months, and then the
circuits will fail due to the corrosion of the liquids. By then, it is
hard to identify the real cause and insurance won't pay.

But if judging devices by unusual happenings is what you do, then
obviously you need the one you can take to bits. Most people will be
fine with a fixed one, as shown by the real life statistics.

Matthew
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 11:22 AM   in response to: Adem Meda in response to: Adem Meda
Am 04.07.2014 03:17, schrieb Adem Meda:
Dalija Prasnikar wrote:

What do you convert that UTF8 text/data to and from?

I don't convert it, I leave it in UTF8 format and do direct processing. If I
would use Delphi unicode string type I would had two unneccessary conversions
that take time + doubled memory requirements (that would mean that processing
is no longer possible on 32-bit Windows I still have to support).

(Disclaimer: Even though I work with a lot of text data, I don't do mobile.)

I don't know what sort of applications you deal with that doubling (or even
quadrubling) the memory used by text data means prohibitively high usage of
RAM, or makes the app too slow.

But personally I have never come across a case where that sort of thing had
any significant effect on my apps.

Hello,

not that I'm doing these kinds of applications, but bio informatics
immediatelly comes to mind! Their alphabet mostly constists of G,C,A,T
and U chars...

And now process the whole human DNA for example...

Greetings

Markus
Adem Meda

Posts: 495
Registered: 12/28/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 4, 2014 5:57 PM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

not that I'm doing these kinds of applications, but bio informatics
immediatelly comes to mind! Their alphabet mostly constists of G,C,A,T
and U chars...

I don't do that sort of stuff either.

But I'd expect anyone who's doing that to use structures other than
human-readable strings.
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2014 11:19 PM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Rudy Velthuis (TeamB) wrote:
Dalija Prasnikar wrote:

Rudy Velthuis (TeamB) wrote:
Quentin Correll wrote:

Rudy,

If a different format is required, a conversion
should be done.

But often a "conversion" is just NOT practical!

Then try to avoid conversions until necessary.

But when you don't have 8-bit strings you cannot avoid conversion,
that's the point.

Depends on what you intend to do with the data. And I wrote "until
necessary", not "forever".

And what when you have situation where conversion should never occur?
I am processing large UTF8 encoded XML files

As I said, I'm sure that people in one-string-type-only languages
manage to do that as well. <shrug>

Again, all the things you guys do with data are very likely done by
others too, and much of this in languages which only have one string
type. They probably also receive single-byte texts, etc. and
probably have to convert. But they seem to manage
very-well-thank-you, so what is wrong with Delphi developers that
they can't?

Will you please stop pushing what other languages have and don't have.

I am not pushing anything, actually. I am merely saying that those
people do manage, so why can't Delphi developers?

--
Rudy Velthuis (TeamB) http://www.teamb.com

"In the end, we will remember not the words of our enemies, but
the silence of our friends."
-- Martin Luther King, Jr.
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 3, 2014 1:35 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
Dalija Prasnikar wrote:

As I said, I'm sure that people in one-string-type-only languages
manage to do that as well. <shrug>


First question is with what effort that manage to do what they need
and how fast their solutions run. Also using of C++ libraries is not
uncommon when you need to squeeze more juice.

With Delphi I don't have to use C++, and I can do what I need fast and
clean. My code is simple and maintainable and it runs within constraints
I have considering speed and memory footprint.

Dalija Prasnikar
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 3, 2014 2:59 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Rudy Velthuis (TeamB) wrote:
Dalija Prasnikar wrote:

As I said, I'm sure that people in one-string-type-only languages
manage to do that as well. <shrug>


First question is with what effort that manage to do what they need
and how fast their solutions run. Also using of C++ libraries is not
uncommon when you need to squeeze more juice.

I doubt they'll have to. Most of the libraries I've seen for such
languages are written in these languages.

With Delphi I don't have to use C++, and I can do what I need fast and
clean. My code is simple and maintainable and it runs within
constraints I have considering speed and memory footprint.

And you do have an UTF-8 string type, so what is the problem? I doubt
you'll be loading such huge amounts of UTF-8 data on a mobile phone.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"When I told the people of Northern Ireland that I was an
atheist, a woman in the audience stood up and said, ëYes, but
is it the God of the Catholics or the God of the Protestants in
whom you don't believe?'"
-- Quentin Crisp
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 3, 2014 4:28 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
Dalija Prasnikar wrote:


With Delphi I don't have to use C++, and I can do what I need fast and
clean. My code is simple and maintainable and it runs within
constraints I have considering speed and memory footprint.

And you do have an UTF-8 string type, so what is the problem? I doubt
you'll be loading such huge amounts of UTF-8 data on a mobile phone.

You do know what library is. It is code you write once and reuse in different
projects. My core library is built on top of UTF8String. That same library has
plenty of code I would need to use on mobile. Because of what I need on
desktop I cannot change my library ever (and such change would take too much
time anyway).

I have already moved to Android Studio and Xcode for mobile development for
apps that are not highly dependent on my Delphi libraries. For apps that need to
reuse my Delphi code I am stuck big time. Most likely I will go FPC + Java +
Swift (Objective-C) for that code, but that will not happen soon (8-bit strings are not
the only reason).

If Delphi introduces 8-bit strings in mobile in next two releases, then my problems
would be solved. If not then it will probably stay forever out of the picture for me.

Dalija Prasnikar
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 3, 2014 6:53 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

You do know what library is. It is code you write once and reuse in
different projects. My core library is built on top of UTF8String.
That same library has plenty of code I would need to use on mobile.
Because of what I need on desktop I cannot change my library ever
(and such change would take too much time anyway).

I have already moved to Android Studio and Xcode for mobile
development for apps that are not highly dependent on my Delphi
libraries.

Why not Delphi for apps that are not dependent on your Delphi
libraries? Anyway, if you use a totally different system, you will have
to rewrite EVERYTHING. And Java also only has one string type. So does
C# (e.g. Xamarin). And also Swift has only one string type (but it can
convert, like Delphi).

Delphi would allow you to write for both mobile OSes, which Android
Studio and Xcode/Swift won't. Do you plan to write everything twice?

I don't see the rationale in your choice, but hey, it is your call.
--
Rudy Velthuis (TeamB) http://www.teamb.com

"To eat well in England, you should have breakfast three times
a day." -- W. Somerset Maugham, 1925
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 3, 2014 7:11 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
Dalija Prasnikar wrote:

You do know what library is. It is code you write once and reuse in
different projects. My core library is built on top of UTF8String.
That same library has plenty of code I would need to use on mobile.
Because of what I need on desktop I cannot change my library ever
(and such change would take too much time anyway).

I have already moved to Android Studio and Xcode for mobile
development for apps that are not highly dependent on my Delphi
libraries.

Why not Delphi for apps that are not dependent on your Delphi
libraries? Anyway, if you use a totally different system, you will have
to rewrite EVERYTHING. And Java also only has one string type. So does
C# (e.g. Xamarin). And also Swift has only one string type (but it can
convert, like Delphi).

1. Delphi for Android doesn't support devices I need to support for my
commercial apps.
2. FireMonkey is still quite buggy.
3. I am not the only developer on mobile projects, because I alone cannot
do things in timeframe needed, because of 1 and 2.

Support for 8-bit strings as such on both platforms is not crucial if I have
to write code from scratch.

Delphi would allow you to write for both mobile OSes, which Android
Studio and Xcode/Swift won't. Do you plan to write everything twice?

I have different projects planned for Android and iOS. Also since, I already
have to use Java for targeting Android, there is very little sense in using
Delphi for iOS only, because of 2 and 3 (and I cannot reuse any existing
Delphi code).

Dalija Prasnikar
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 3, 2014 7:35 AM   in response to: Dalija Prasnikar in response to: Dalija Prasnikar
Dalija Prasnikar wrote:

Delphi would allow you to write for both mobile OSes, which Android
Studio and Xcode/Swift won't. Do you plan to write everything twice?

I have different projects planned for Android and iOS.

Really? Not one app (in different eiditions, of course) for both
platforms, as is usual, these days?

Also since, I
already have to use Java for targeting Android

You DO know that Java only has one string type, right? <g>

And Delphi does not require you to use Java, only to interface with it,
here and there.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Sanity is a madness put to good uses."
-- George Santayana (1863-1952)
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 3, 2014 10:36 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 03.07.2014 16:35, schrieb Rudy Velthuis (TeamB):
Dalija Prasnikar wrote:

Delphi would allow you to write for both mobile OSes, which Android
Studio and Xcode/Swift won't. Do you plan to write everything twice?

I have different projects planned for Android and iOS.

Really? Not one app (in different eiditions, of course) for both
platforms, as is usual, these days?

Also since, I
already have to use Java for targeting Android

You DO know that Java only has one string type, right? <g>

And Delphi does not require you to use Java, only to interface with it,
here and there.

You are able of properly reading, right?
It was already written that those apps aren't using the mentioned "core"
libraries and thus aren't really dependant on strings.

Greetingsa

Markus
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 8:38 AM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

You DO know that Java only has one string type, right? <g>

And Delphi does not require you to use Java, only to interface with
it, here and there.

You are able of properly reading, right?

Was that a rhetorical question?

It was already written that those apps aren't using the mentioned
"core" libraries and thus aren't really dependant on strings.

Then, if strings don't matter, the fact that Delphi for mobile has only
one 16-bit string type can hardly be an argument against it.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"We, on our side, are praying to Him to give us victory,
because we believe we are right; but those on the other side
pray to Him, too, for victory, believing they are right. What
must He think of us?"
-- Abraham Lincoln
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 2:27 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 05.07.2014 17:38, schrieb Rudy Velthuis (TeamB):
Markus Humm wrote:

You DO know that Java only has one string type, right? <g>

And Delphi does not require you to use Java, only to interface with
it, here and there.

You are able of properly reading, right?

Was that a rhetorical question?

It was already written that those apps aren't using the mentioned
"core" libraries and thus aren't really dependant on strings.

Then, if strings don't matter, the fact that Delphi for mobile has only
one 16-bit string type can hardly be an argument against it.

Hello,

now I'm convinced you didn't properly following the conversation. Sorry
to say!

It was stated that there are 2 kinds of mobile applications:

1. those who would need to use these 8 bit string based core libraries.
Those cannot be properly done with Delphi as is.

2. some other small scale apps which are not needing those libraries,
which could be done with Delphi but which aren't too important.

So now tell me why 8 bit based strings don't matter here?

Greetings

Markus
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 11:44 AM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

Am 05.07.2014 17:38, schrieb Rudy Velthuis (TeamB):
Markus Humm wrote:

You DO know that Java only has one string type, right? <g>

And Delphi does not require you to use Java, only to interface
with >>> it, here and there.

You are able of properly reading, right?

Was that a rhetorical question?

It was already written that those apps aren't using the mentioned
"core" libraries and thus aren't really dependant on strings.

Then, if strings don't matter, the fact that Delphi for mobile has
only one 16-bit string type can hardly be an argument against it.

Hello,

now I'm convinced you didn't properly following the conversation.
Sorry to say!

It was stated that there are 2 kinds of mobile applications:

1. those who would need to use these 8 bit string based core
libraries. Those cannot be properly done with Delphi as is.

But of course they can be done properly with Delphi as-is! Take a look
at the RTL and see how Embarcadero do it. They do use these 8-bit
libraries here and there, and you can do that too. That is why there
are types like MarshaledAString (which is, in fact, a PAnsiChar), etc.

2. some other small scale apps which are not needing those libraries,
which could be done with Delphi but which aren't too important.

So now tell me why 8 bit based strings don't matter here?

Because the internals of the string type do not matter, as long as
there is a possibility to convert it to (and from) what is necessary to
be able to interface with external code.

But on Android, the basic string type is UTF-16, and on iOS, it is
UTF-16 too. There are 8-bit-char routines in some low level APIs, but
you handle that the same way as Java or Objective-C would do that, and
you are not very likely to need those APIs very often.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Why yes -- a bulletproof vest."
-- James Rodges, murderer, on his final request before the
firing squad.
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 15, 2014 1:09 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 06.07.2014 20:44, schrieb Rudy Velthuis (TeamB):
Markus Humm wrote:


Hello,

now I'm convinced you didn't properly following the conversation.
Sorry to say!

It was stated that there are 2 kinds of mobile applications:

1. those who would need to use these 8 bit string based core
libraries. Those cannot be properly done with Delphi as is.

But of course they can be done properly with Delphi as-is! Take a look
at the RTL and see how Embarcadero do it. They do use these 8-bit
libraries here and there, and you can do that too. That is why there
are types like MarshaledAString (which is, in fact, a PAnsiChar), etc.

Hello,

here's your old idea again: reinvent the wheel. But why? it was there
already and afaik even in all pascal code already. Why not recycle this
tried and tested stuff?

Just add it as some sample unit in the samples collection and add a
comment that it's unsupported and everybody will be lucky I guess.
(except maybe the installer designer as he has to add some files)


2. some other small scale apps which are not needing those libraries,
which could be done with Delphi but which aren't too important.

So now tell me why 8 bit based strings don't matter here?

Because the internals of the string type do not matter, as long as
there is a possibility to convert it to (and from) what is necessary to
be able to interface with external code.

But on Android, the basic string type is UTF-16, and on iOS, it is
UTF-16 too. There are 8-bit-char routines in some low level APIs, but
you handle that the same way as Java or Objective-C would do that, and
you are not very likely to need those APIs very often.

Conversion is all nice, but why even convert data (often stemming from
internet protocols)
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 15, 2014 3:13 PM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

1. those who would need to use these 8 bit string based core
libraries. Those cannot be properly done with Delphi as is.

But of course they can be done properly with Delphi as-is! Take a
look at the RTL and see how Embarcadero do it. They do use these
8-bit libraries here and there, and you can do that too. That is
why there are types like MarshaledAString (which is, in fact, a
PAnsiChar), etc.

Hello,

here's your old idea again: reinvent the wheel.

Who is reinventing the wheel? The thing simply got a different name.
Let's call it "Rad" or "Wiel".

Fact is that you can easily communicate with C-char (8-bit char) based
APIs, even in the mobile compilers. Fact is, too, that you are not
really encouraged to do it, unless you know what you are doing. You are
encouraged to use higher level APIs instead.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"I have a spelling checker,
It came with my PC;
It plainly marks four my revue
Mistakes I cannot sea.
I've run this poem threw it,
I'm sure your pleased too no,
Its letter perfect in it's weigh,
My checker tolled me sew."
-- Janet Minor, "Spellbound"
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 16, 2014 1:21 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 16.07.2014 00:13, schrieb Rudy Velthuis (TeamB):
Markus Humm wrote:

1. those who would need to use these 8 bit string based core
libraries. Those cannot be properly done with Delphi as is.

But of course they can be done properly with Delphi as-is! Take a
look at the RTL and see how Embarcadero do it. They do use these
8-bit libraries here and there, and you can do that too. That is
why there are types like MarshaledAString (which is, in fact, a
PAnsiChar), etc.

Hello,

here's your old idea again: reinvent the wheel.

Who is reinventing the wheel? The thing simply got a different name.
Let's call it "Rad" or "Wiel".

Fact is that you can easily communicate with C-char (8-bit char) based
APIs, even in the mobile compilers. Fact is, too, that you are not
really encouraged to do it, unless you know what you are doing. You are
encouraged to use higher level APIs instead.

Hello,

it's not about using 8 bit based APIs.

Reinventing the wheel means: 8 bit based strings were already
implemented as pure pascal solution afaik and those who need it
on a mobile platform are forced to "reinvent" (or better create ersatz
for them) now. I'd consider that a waste of time if I had the need for
such strings due to memory constraints and the need to use big data best
dealt with as string.

Greetings

Markus
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 16, 2014 11:12 PM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

Am 16.07.2014 00:13, schrieb Rudy Velthuis (TeamB):
Markus Humm wrote:

1. those who would need to use these 8 bit string based core
libraries. Those cannot be properly done with Delphi as is.

But of course they can be done properly with Delphi as-is! Take a
look at the RTL and see how Embarcadero do it. They do use these
8-bit libraries here and there, and you can do that too. That is
why there are types like MarshaledAString (which is, in fact, a
PAnsiChar), etc.

Hello,

here's your old idea again: reinvent the wheel.

Who is reinventing the wheel? The thing simply got a different name.
Let's call it "Rad" or "Wiel".

Fact is that you can easily communicate with C-char (8-bit char)
based APIs, even in the mobile compilers. Fact is, too, that you
are not really encouraged to do it, unless you know what you are
doing. You are encouraged to use higher level APIs instead.

Hello,

it's not about using 8 bit based APIs.

Reinventing the wheel means: 8 bit based strings were already
implemented as pure pascal solution afaik and those who need it
on a mobile platform are forced to "reinvent" (or better create ersatz
for them) now.

If you coded in a portable way, you should not need them. You only need
the pointer types in order to interface with APIs. If you receive such
data (say, UTF-8) from another source, simply convert it to the
available string type.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Many people would sooner die than think; in fact, they
do so." -- Bertrand Russell
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 17, 2014 1:29 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 17.07.2014 08:12, schrieb Rudy Velthuis (TeamB):
Markus Humm wrote:


Hello,

it's not about using 8 bit based APIs.

Reinventing the wheel means: 8 bit based strings were already
implemented as pure pascal solution afaik and those who need it
on a mobile platform are forced to "reinvent" (or better create ersatz
for them) now.

If you coded in a portable way, you should not need them. You only need
the pointer types in order to interface with APIs. If you receive such
data (say, UTF-8) from another source, simply convert it to the
available string type.

Hello,

I was talking about those cases where somebody (e.g. due to memory
constraints) wants to keep the data he's loading in 8-bit strings.
I do know that I could convert to the available string type otherwise.
It's simply for those few occassions where an 8 bit string would be best
fit. Nothing more and nothing less. And no need to turn the case into
some other direction.

Greetings

Markus
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 19, 2014 11:46 AM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

Am 17.07.2014 08:12, schrieb Rudy Velthuis (TeamB):
Markus Humm wrote:


Hello,

it's not about using 8 bit based APIs.

Reinventing the wheel means: 8 bit based strings were already
implemented as pure pascal solution afaik and those who need it
on a mobile platform are forced to "reinvent" (or better create
ersatz >> for them) now.

If you coded in a portable way, you should not need them. You only
need the pointer types in order to interface with APIs. If you
receive such data (say, UTF-8) from another source, simply convert
it to the available string type.

Hello,

I was talking about those cases where somebody (e.g. due to memory
constraints) wants to keep the data he's loading in 8-bit strings.

And I have already repeatedly said that a design which needs to use
8-bit strings due to memory constraints is rather fragile, and
certainly not scalable.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Truthful words are not beautiful; beautiful words are not
truthful. Good words are not persuasive; persuasive words are
not good."
-- Lao tzu
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 19, 2014 3:42 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy,

| And I have already repeatedly said that a design which needs to use
| 8-bit strings due to memory constraints is rather fragile, and
| certainly not scalable.

Come on Rudy,... scalability has absolutely nothing to do with 8-bit
strings. Many projects that use 8-bit strings can scale better than
those using wide/Unicode strings!!!

--

Q

07/19/2014 15:40:15

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 19, 2014 3:48 PM   in response to: Quentin Correll in response to: Quentin Correll
Quentin Correll wrote:

Rudy,

And I have already repeatedly said that a design which needs to use
8-bit strings due to memory constraints is rather fragile, and
certainly not scalable.

Come on Rudy,... scalability has absolutely nothing to do with 8-bit
strings.

If the app would run out of memory if the string type changed, it has
to do with scalability.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"We must all hear the universal call to like your neighbor like
you like to be liked yourself." -- George W. Bush
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 19, 2014 7:13 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy,

| If the app would run out of memory if the string type changed, it has
| to do with scalability.

YES!!! And that means wider strings do NOT scale as well!!!!!!!

--

Q

07/19/2014 19:12:52

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 16, 2014 11:06 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:

But of course they can be done properly with Delphi as-is! Take a
look at the RTL and see how Embarcadero do it. They do use these
8-bit libraries here and there, and you can do that too. That is
why there are types like MarshaledAString (which is, in fact, a
PAnsiChar), etc.

here's your old idea again: reinvent the wheel.

Who is reinventing the wheel? The thing simply got a different name.
Let's call it "Rad" or "Wiel".

Fact is that you can easily communicate with C-char (8-bit char) based
APIs, even in the mobile compilers. Fact is, too, that you are not
really encouraged to do it, unless you know what you are doing. You
are encouraged to use higher level APIs instead.

And these higher level APIs use UTF-16 throughout.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"If people are good only because they fear punishment, and hope for
reward, then we are a sorry lot indeed." -- Albert Einstein
Markus Humm

Posts: 5,113
Registered: 11/9/03
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 17, 2014 1:30 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Am 17.07.2014 08:06, schrieb Rudy Velthuis (TeamB):
Rudy Velthuis (TeamB) wrote:

But of course they can be done properly with Delphi as-is! Take a
look at the RTL and see how Embarcadero do it. They do use these
8-bit libraries here and there, and you can do that too. That is
why there are types like MarshaledAString (which is, in fact, a
PAnsiChar), etc.

here's your old idea again: reinvent the wheel.

Who is reinventing the wheel? The thing simply got a different name.
Let's call it "Rad" or "Wiel".

Fact is that you can easily communicate with C-char (8-bit char) based
APIs, even in the mobile compilers. Fact is, too, that you are not
really encouraged to do it, unless you know what you are doing. You
are encouraged to use higher level APIs instead.

And these higher level APIs use UTF-16 throughout.

Hello,

maybe but that's irrelevaant here. I was not talking about API access,
and I won't talk about API access.

Greetings

Markus
Dalija Prasnikar

Posts: 2,325
Registered: 11/9/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 3, 2014 1:38 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
Dalija Prasnikar wrote:

Delphi would allow you to write for both mobile OSes, which Android
Studio and Xcode/Swift won't. Do you plan to write everything twice?

I have different projects planned for Android and iOS.

Really? Not one app (in different eiditions, of course) for both
platforms, as is usual, these days?

For start no. Completely different apps for different platforms.

And FYI I know the value of having single code base, just that is not
the only thing that matters when deciding which tool will be used for
particular project.

Also since, I
already have to use Java for targeting Android

You DO know that Java only has one string type, right? <g>

Nooooo, really......

Of course, I know. It just doesn't matter too much for the apps I am
writing right now.

And Delphi does not require you to use Java, only to interface with it,
here and there.

I need to target SDK on Android and not NDK, and I need to support
devices from API 9 up. So Delphi is of no use here, even if it would have
8-bit strings.

However for some in-house projects I could target NDK and use Delphi, but
for those projects reusing my code is crucial. We are also considering using
Windows tablets for those apps.

Dalija Prasnikar
Arthur Hoornweg

Posts: 414
Registered: 6/2/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 4:19 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:

Delphi would allow you to write for both mobile OSes, which Android
Studio and Xcode/Swift won't. Do you plan to write everything twice?

Re-using existing code is what makes Delphi attractive for mobile. If that option is reduced by removing often-used features (such as 8-bit strings) from Delphi then it becomes less attractive. It makes sense to maximize attractiveness, not minimize it.

Bruce McGee

Posts: 1,716
Registered: 9/30/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 4:42 AM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

Re-using existing code is what makes Delphi attractive for mobile. If
that option is reduced by removing often-used features (such as 8-bit
strings) from Delphi then it becomes less attractive. It makes sense
to maximize attractiveness, not minimize it.

Exactly my posiion on this.

Even if this won't affect me (much?) and I doubt it will be that big a
deal for most mobile developers.

--
Regards,
Bruce McGee
Glooscap Software
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 8:38 AM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

Rudy Velthuis (TeamB) wrote:

Delphi would allow you to write for both mobile OSes, which Android
Studio and Xcode/Swift won't. Do you plan to write everything twice?

Re-using existing code is what makes Delphi attractive for mobile. If
that option is reduced by removing often-used features (such as 8-bit
strings) from Delphi then it becomes less attractive.


Not if you already wrote your code (or refactor it now) in a more
portable way.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Once all struggle is grasped, miracles are possible."
-- Mao Tse Tung
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 5, 2014 10:14 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy,

| Not if you already wrote your code (or refactor it now) in a more
| portable way.

For some apps that's just not possible!

--

Q

07/05/2014 10:14:02

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 11:44 AM   in response to: Quentin Correll in response to: Quentin Correll
Quentin Correll wrote:

Rudy,

Not if you already wrote your code (or refactor it now) in a more
portable way.

For some apps that's just not possible!

It is possible. It may not be worthwhile, or simply too much work, but
it is almost certainly possible.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"You can pretend to be serious; you can't pretend to be witty."
-- Sacha Guitry (1885-1957)
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2014 2:48 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy,

| | For some apps that's just not possible!
|
| It is possible. It may not be worthwhile, or simply too much work, but
| it is almost certainly possible.

We'll have to disagree. There are some apps where it is impossible!

Such as the neural simulation apps, such as run on Blue Gene. <shrug>

--

Q

07/06/2014 14:40:58

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 10:01 AM   in response to: Quentin Correll in response to: Quentin Correll
Quentin Correll wrote:

Rudy,

| For some apps that's just not possible!

It is possible. It may not be worthwhile, or simply too much work,
but it is almost certainly possible.

We'll have to disagree. There are some apps where it is impossible!

I don't think it is impossible. Just not worth the trouble and time,
perhaps. But possible.

Such as the neural simulation apps, such as run on Blue Gene.

Huh? Why do neural simulations require AnsiChars and AnsiStrings at all?
--
Rudy Velthuis (TeamB) http://www.teamb.com

"Of all the enemies to public liberty, war is perhaps the most
to be dreaded because it comprises and develops the germ of
every other." -- James Madison
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 10:51 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy,

| | Such as the neural simulation apps, such as run on Blue Gene.
|
| Huh? Why do neural simulations require AnsiChars and AnsiStrings at
| all?

I didn't say they needed strings, just an app that is impossible to
convert to the mobile environment.

--

Q

07/07/2014 10:49:56

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 11:26 PM   in response to: Quentin Correll in response to: Quentin Correll
Quentin Correll wrote:

Rudy,

| Such as the neural simulation apps, such as run on Blue Gene.

Huh? Why do neural simulations require AnsiChars and AnsiStrings at
all?

I didn't say they needed strings, just an app that is impossible to
convert to the mobile environment.

Oh, Ok, that may be.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"When I am working on a problem I never think about beauty. I
only think about how to solve the problem. But when I have
finished, if the solution is not beautiful, I know it is wrong."
-- Buckminster Fuller (1895-1983)

Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 8, 2014 12:20 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy,

| | I didn't say they needed strings, just an app that is impossible to
| | convert to the mobile environment.

| Oh, Ok, that may be.

<chuckle> ;-)

--

Q

07/08/2014 12:20:16

1.19.1.372 [Q'sBrokenToolBar] [Running on TQ]

Arthur Hoornweg

Posts: 414
Registered: 6/2/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 7:01 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:
Arthur Hoornweg wrote:

Not if you already wrote your code (or refactor it now) in a more
portable way.

8-bit strings used to be the ONLY Pascal string type for a decade and a half, so please forgive us for being so short-sighted to use them. And since it is impossible to predict what features are going to be removed from a language sometime in the future, it's only possible to write "portable" software with perfect hindsight. Who knows, 16-bit "Unicodestring" may well become deprecated some day in favor of a 32-bit version. After all, the Unicode code point space really is 32 bit, not 16. How silly for us to assume a widechar to be something as "narrow" as 16-bit. Surely we should have realized from the very beginning that this variable type has no future and we should have written portable code instead.

Bruce McGee

Posts: 1,716
Registered: 9/30/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 7:07 AM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

Rudy Velthuis (TeamB) wrote:
Arthur Hoornweg wrote:

Not if you already wrote your code (or refactor it now) in a more
portable way.

8-bit strings used to be the ONLY Pascal string type for a decade and
a half, so please forgive us for being so short-sighted to use them.


And for about half of that time, we were warned against writing code
that relies on the length of a character.

It's not like this all happened out of thin air.

--
Regards,
Bruce McGee
Glooscap Software
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 7, 2014 10:06 AM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

Rudy Velthuis (TeamB) wrote:
Arthur Hoornweg wrote:

Not if you already wrote your code (or refactor it now) in a more
portable way.

8-bit strings used to be the ONLY Pascal string type for a decade and
a half

And before that, 256 byte strings were the only Pascal string types for
a long time. <shrug>

Things evolve. Write your code thus that it does not rely on knowledge
of internals at all, and it is portable. Use system routines and
constants to find out some of the characteristics you must know, like
the size of a float, size of an integer, endianness, size of a char,
encoding, thousands separator, etc.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"Everything of importance has been said before by somebody who
did not discover it."
-- Alfred North Whitehead
Arthur Hoornweg

Posts: 414
Registered: 6/2/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 8, 2014 7:03 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:

Things evolve. Write your code thus that it does not rely on knowledge
of internals at all, and it is portable. Use system routines and
constants to find out some of the characteristics you must know, like
the size of a float, size of an integer, endianness, size of a char,
encoding, thousands separator, etc.

Intimate knowledge of string types is still necessary because system routines like length(), copy() and delete() weren't designed to work with "characters" in the common sense of "letters". An expression like COPY (name,1,3) isn't guaranteed to return the first three letters of a person's name, it fails with some languages. Also, the size of floats and integers directly influences the range and precision of the values that you can store in them. These have to match the range and precision of any database fields you want to store them in or you lose accuracy. So knowledge of internals is essential.

I am very much against "dumbing down" the language. Nobody is forced to use features he or she doesn't need. It's not like they're in the way or something.

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 8, 2014 7:21 AM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

Rudy Velthuis (TeamB) wrote:

Things evolve. Write your code thus that it does not rely on
knowledge of internals at all, and it is portable. Use system
routines and constants to find out some of the characteristics you
must know, like the size of a float, size of an integer,
endianness, size of a char, encoding, thousands separator, etc.

Intimate knowledge of string types is still necessary because system
routines like length(), copy() and delete() weren't designed to work
with "characters" in the common sense of "letters".


You must know that if the string is UTF-16, or converted to it, it can
have surrogate pairs, etc. but that has nothing to do with compile time
knowledge of the internals.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"You can pretend to be serious; you can't pretend to be witty."
-- Sacha Guitry (1885-1957)
Arthur Hoornweg

Posts: 414
Registered: 6/2/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 10, 2014 6:11 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:

You must know that if the string is UTF-16, or converted to it, it can
have surrogate pairs, etc. but that has nothing to do with compile time
knowledge of the internals.

Well, to me "portable code" is not just code that compiles without warnings on all platforms, it should actually work correctly. Also, there's an ugly confusion between a "character" in the sense of a written glyph that's an element of a human-language word and a "char" in the sense of an element of a computer string. Even the Delphi help text uses the terms interchangeably, pretending that "Copy() returns a substring or subarray containing Count characters ".

If it really is your opinion that a language should have only one string type, then we might as well get the version that works as advertized. Which happens to be UCS-4. No more confusion. Ever.

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 12, 2014 1:37 PM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

Rudy Velthuis (TeamB) wrote:

You must know that if the string is UTF-16, or converted to it, it
can have surrogate pairs, etc. but that has nothing to do with
compile time knowledge of the internals.

Well, to me "portable code" is not just code that compiles without
warnings on all platforms, it should actually work correctly.

Duh!

Not sure how that invalidates what I said.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"There Ought to be Limits to Freedom!"
-- George W. Bush, commenting on gwbush.com (05/21/1999)
Arthur Hoornweg

Posts: 414
Registered: 6/2/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 13, 2014 11:44 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:

Duh!

Not sure how that invalidates what I said.

That actually happened to me quite often in the past few years. Having open-source Delphi code that was alledgedly "unicodified" and compiled without warning under all Delphi versions, yet behaved differently when compiled with D2009+. Simply because "under the hood" some assumptions about character size and codepage-agnosticism were still lurking.

For the rest I'm all for portable code and I also like the concept of ARC. But I'd really prefer it if speed-optimized ASM versions of routines in the RTL/VCL weren't replaced by slower "pure pascal" routines. Nothing wrong with a couple of IFDEFS in the RTL/VCL.

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 15, 2014 10:17 AM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

Rudy Velthuis (TeamB) wrote:

Duh!

Not sure how that invalidates what I said.

That actually happened to me quite often in the past few years.
Having open-source Delphi code that was alledgedly "unicodified" and
compiled without warning under all Delphi versions, yet behaved
differently when compiled with D2009+. Simply because "under the
hood" some assumptions about character size and codepage-agnosticism
were still lurking.


Well, no one ever said you could simply recompile. Only if code was
actually portable, you could.
--
Rudy Velthuis (TeamB) http://www.teamb.com

"I doubt, therefore I might be." -- Unknown
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 12, 2014 1:47 PM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

Even the Delphi help text uses the terms interchangeably, pretending
that "Copy() returns a substring or subarray containing Count
characters".

Well, I assume that text is pretty old already, and it was true when it
was written.
--
Rudy Velthuis (TeamB) http://www.teamb.com

"Java: the elegant simplicity of C++ and the blazing speed of
Smalltalk." -- Roland Turner
Arthur Hoornweg

Posts: 414
Registered: 6/2/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 13, 2014 11:30 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:

Well, I assume that text is pretty old already, and it was true when it
was written.

True only for a small subset of languages. The rest of the world needed mbcs.
Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 14, 2014 6:33 AM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

Rudy Velthuis (TeamB) wrote:

Well, I assume that text is pretty old already, and it was true
when it was written.

True only for a small subset of languages.

It was true for ASCII (*American* Standard Code for Information
Interchange). That was in the days when AnsiStrings did not carry their
own codepages with them yet. <g>

--
Rudy Velthuis (TeamB) http://www.teamb.com

"One's mind, once stretched by a new idea, never regains its
original dimensions."
-- Oliver Wendell Holmes
Arthur Hoornweg

Posts: 414
Registered: 6/2/98
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 14, 2014 11:59 PM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy Velthuis (TeamB) wrote:

It was true for ASCII (*American* Standard Code for Information
Interchange). That was in the days when AnsiStrings did not carry their
own codepages with them yet. <g>

That would have been true if Pascal/Delphi had ever implemented 7-bit ASCII strings (or simply masked the 8th bit) but that has never been the case. Delphi/Pascal strings have always been at least 8-bit wide and the definition of the upper 128 chars was up to the operating system. Therefore, there never really was a time where "chars" in Delphi/Pascal had a bijective 1:1 relationship to "characters" but then again, it wasn't the fault of the programming language.

AFAIK, Windows (and hence, Delphi) has always had code pages. MS/DOS also had code pages. Not even CP/M was totally ASCII, the upper 128 characters were implemented in various ways by various manufacturers.

To my surprise, Turbo Pascal for CP/M still has a fan base on the internet. This awesome little gem is what got me hooked to Pascal in my university days :
http://techtinkering.com/2013/03/05/turbo-pascal-a-great-choice-for-programming-under-cpm/

Rudy Velthuis (...


Posts: 7,731
Registered: 9/22/99
Re: Plea from Scientific and Engineering users
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 15, 2014 8:42 AM   in response to: Arthur Hoornweg in response to: Arthur Hoornweg
Arthur Hoornweg wrote:

Rudy Velthuis (TeamB) wrote:

It was true for ASCII (*American* Standard Code for Information
Interchange). That was in the days when AnsiStrings did not carry
their own codepages with them yet. <g>

That would have been true if Pascal/Delphi had ever implemented 7-bit
ASCII strings (or simply masked the 8th bit) but that has never been
the case. Delphi/Pascal strings have always been at least 8-bit wide


Sure, but the codepage was never stored with the text. How you
interpreted these 8bits was up to you.

--
Rudy Velthuis (TeamB) http://www.teamb.com

"DOS Computers manufactured by companies such as IBM, Compaq,
Tandy, and millions of others are by far the most popular, with
about 70 million machines in use worldwide. Macintosh fans, on
the other hand, may note that cockroaches are far more numerous
than humans, and that numbers alone do not denote a higher life
form." -- New York Times, November 26, 1991