Watch, Follow, &
Connect with Us

Welcome, Guest
Guest Settings
Help

Thread: Inconsistent throwing of exceptions


This question is answered. Helpful answers available: 0. Correct answers available: 1.


Permlink Replies: 25 - Last Post: Aug 8, 2017 8:24 AM Last Post By: Markus Humm
Enquiring Mind

Posts: 87
Registered: 10/6/08
Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 21, 2017 9:14 AM
Hi,

An application I am currently developing with Delphi 7 has been functioning satisfactorily for the last few weeks, but has recently started intermittently throwing an "out of memory" exception at a random point within the data loading procedure. That is to say, when repeatedly running the same code to read the same data file from disc, sometimes the procedure runs without error, and sometimes it throws an "out of memory" exception when reading an object in a collection of around 700 objects which can be different from time to time. An example of a recent exception error message is:

Error reading TBookColn item ID 690 Token= "C" (Error in = TBookBase.ReadFromTextStream for ID= 690 (Error in GetNextToken: PrevToken= "," CurrToken= "C" CurrLine=", Subject: 184, CategorySet: "[75, 144, 180]", Location: 49, BookType: "Soft cover", ], " (Out of memory)))

The data file being read has been getting bigger, and the memory footprint of the global object has been also been increasing in size as data is added to the data structure, but the size of each is still quite low in relation to the capacity of the computer.

Can anyone suggest what sort of problem could be causing these sporadic "out of memory" exceptions?

TIA,
EM
Alex Belo

Posts: 535
Registered: 10/8/06
Re: Inconsistent throwing of exceptions
Helpful
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 21, 2017 9:30 AM   in response to: Enquiring Mind in response to: Enquiring Mind
Enquiring Mind wrote:

Can anyone suggest what sort of problem could be causing these
sporadic "out of memory" exceptions?

1) memory fragmentation
2) CodeGuard (but this is BCB beast).
3) some bug in your code...

--
Alex
Enquiring Mind

Posts: 87
Registered: 10/6/08
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 22, 2017 3:20 AM   in response to: Alex Belo in response to: Alex Belo
Alex Belo wrote:
Enquiring Mind wrote:

Can anyone suggest what sort of problem could be causing these
sporadic "out of memory" exceptions?

1) memory fragmentation
2) CodeGuard (but this is BCB beast).
3) some bug in your code...
Thanks for your quick response. It seems that (1) and (3) are the most likely. Not least, because CodeGuard is something I know nothing about. In the case of (3), rather than being a case of a bug in the code, it's more likely a case of a data structure or algorithm used in the code that whilst not in error, is not optimal or appropriate for the size (ordinality) of the problem. If I eventually trace the detailed causes, I will share them in this thread.

The problem I have been experiencing that if, after launching the application from the executable, it throws an "out of memory" exception while reading the data file, when I then re-run it in the debugger and break the execution at the step where the exception occurred, and then manually step through this code, the exception no longer occurs, and the whole data file is read successfully and transformed into a global memory object.

EM
Alex Belo

Posts: 535
Registered: 10/8/06
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 22, 2017 8:30 AM   in response to: Enquiring Mind in response to: Enquiring Mind
Enquiring Mind wrote:

CodeGuard is something I know nothing about.

This is BCB tool for detecting problems with memory (leaks, buffers
overflows, etc.). It is quite good tool but after introduction of
multicore CPUs problems with real (on parallel cores) multithreading
was found.

The problem I have been experiencing that if, after launching the
application from the executable, it throws an "out of memory"
exception while reading the data file, when I then re-run it in the
debugger and break the execution at the step where the exception
occurred, and then manually step through this code, the exception no
longer occurs

Is this exe compiled in the same "mode"? Exe compiled in debug and
release modes can work differently...

Are there threads involved? Breakpoint can simulate some
"synchronization" in wrongly sinchronized code. You can try setting
affinity of application to 1 core and test.

--
Alex
Roy Lambert

Posts: 959
Registered: 10/21/99
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 22, 2017 11:02 PM   in response to: Alex Belo in response to: Alex Belo
Alex

Are there threads involved? Breakpoint can simulate some
"synchronization" in wrongly sinchronized code. You can try setting
affinity of application to 1 core and test.

In D2006, don't know about later versions, I've also found that having $D- in units can affect where the error is indicated to be. Whan a function / procedure from that unit is used often the error is shown a line or two away from the function / procedure call.

Roy
Roy Lambert

Posts: 959
Registered: 10/21/99
Re: Inconsistent throwing of exceptions
Helpful
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 21, 2017 10:06 AM   in response to: Enquiring Mind in response to: Enquiring Mind
As Alex says you'll get memory fragmentation. You also have to remember that you don't get all of the memory the computer has whilst using D7 - I can't rember how much.

This <<but the size of each is still quite low in relation to the capacity of the computer>> make me wonder are you freeing the objects after you've loaded / used them or are you just creating a new object each time and never freeing it.

I don't think it will apply here but on the recent (last 3 - 4 years) occasions I've had an out of memory error its been a bad jpg.

You could add functionality to monitor the memory before and after loading each item. Have a look at

https://stackoverflow.com/questions/437683/how-to-get-the-memory-used-by-a-delphi-program


Roy Lambert

Rudy Velthuis (...


Posts: 7,469
Registered: 9/22/99
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 21, 2017 2:24 PM   in response to: Roy Lambert in response to: Roy Lambert
Roy Lambert wrote:

As Alex says you'll get memory fragmentation.

Alex mentions it as one of a few possibilities. Another one is not very
unlikely either: a bug in his (EM's) code.

--
Rudy Velthuis http://www.rvelthuis.de

"Programming is like kicking yourself in the face, sooner or
later your nose will bleed."
-- Kyle Woodbury
Enquiring Mind

Posts: 87
Registered: 10/6/08
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 22, 2017 3:28 AM   in response to: Roy Lambert in response to: Roy Lambert
Roy Lambert wrote:
As Alex says you'll get memory fragmentation. You also have to remember that you don't get all of the memory the computer has whilst using D7 - I can't rember how much.

This <<but the size of each is still quite low in relation to the capacity of the computer>> make me wonder are you freeing the objects after you've loaded / used them or are you just creating a new object each time and never freeing it.

I don't think it will apply here but on the recent (last 3 - 4 years) occasions I've had an out of memory error its been a bad jpg.

You could add functionality to monitor the memory before and after loading each item. Have a look at

https://stackoverflow.com/questions/437683/how-to-get-the-memory-used-by-a-delphi-program

The size of the data file, which is simply a text stream representation of the memory global object, is only about 400 kB, tiny in relation to the RAM capacity of the computer. Assuming that the size of the memory object that is read from this is 10 times the data file size, that would make the memory object about 4 MB, still well within the capacity of the computer's RAM. But I must admit that there were several other applications active at time that I launched my application, including Internet Explorer which may be a memory hog for all I know.

Thanks for the link to discussion group thread about getting the used and available memory in Delphi.

EM
Roy Lambert

Posts: 959
Registered: 10/21/99
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 22, 2017 7:56 AM   in response to: Enquiring Mind in response to: Enquiring Mind
EM

The size of the data file, which is simply a text stream representation of the memory global object, is only about 400 kB, tiny in relation to the RAM capacity of the computer. Assuming that the size of the memory object that is read from this is 10 times the data file size, that would make the memory object about 4 MB, still well within the capacity of the computer's RAM. But I must admit that there were several other applications active at time that I launched my application, including Internet Explorer
which may be a memory hog for all I know.

Multiple applications shouldn't be a problem - RAM & the swap file should allow those to be handled. What counts is the memory allocated to your application and the maximum that can be used. From what you've posted so far I can still only make two suggestions. The first is because I don't know what you're doing in your code and is that you create the object, read in from disk, process, create another object read in from disk, process - rinse & repeat but you never free any objects (or possibly a part of the object) so they all hang around in memory. I'd guess c5000 objects would be enough to do the damage. The second guess is the data is corrupt, damaged or malformed in some way which causes something to attempt to be written outside of the available memory.

A third possibility has just occurred to me - does your processing call any 3rd party code? If so the problem could be in there.

Roy Lambert
Alex Belo

Posts: 535
Registered: 10/8/06
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 22, 2017 8:10 AM   in response to: Enquiring Mind in response to: Enquiring Mind
Enquiring Mind wrote:

The size of the data file, which is simply a text stream
representation of the memory global object, is only about 400 kB,
tiny in relation to the RAM capacity of the computer.

You need to compare with the biggest free contiguous memory block of
virtual memory, not with 2Gb of available virtual memory or with RAM
size.

--
Alex
Markus Humm

Posts: 4,816
Registered: 11/9/03
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 21, 2017 11:28 AM   in response to: Enquiring Mind in response to: Enquiring Mind
Am 21.07.2017 um 18:14 schrieb Enquiring Mind:
Hi,

An application I am currently developing with Delphi 7 has been functioning satisfactorily for the last few weeks, but has recently started intermittently throwing an "out of memory" exception at a random point within the data loading procedure. That is to say, when repeatedly running the same code to read the same data file from disc, sometimes the procedure runs without error, and sometimes it throws an "out of memory" exception when reading an object in a collection of around 700 objects which can be d
ifferent from time to time. An example of a recent exception error message is:

Error reading TBookColn item ID 690 Token= "C" (Error in = TBookBase.ReadFromTextStream for ID= 690 (Error in GetNextToken: PrevToken= "," CurrToken= "C" CurrLine=", Subject: 184, CategorySet: "[75, 144, 180]", Location: 49, BookType: "Soft cover", ], " (Out of memory)))

The data file being read has been getting bigger, and the memory footprint of the global object has been also been increasing in size as data is added to the data structure, but the size of each is still quite low in relation to the capacity of the computer.

Can anyone suggest what sort of problem could be causing these sporadic "out of memory" exceptions?

TIA,
EM

Hello,

it might be that the memory manager used in D7 programs is more prone to
memory fragmentation than the FastMM4 used since D2006.

You might want to download FastMM4 and test it. Use should be simple:
place the files in your source directory and add FastMM4 as first unit
in the uses of your .dpr file.

Greetings

Markus
Enquiring Mind

Posts: 87
Registered: 10/6/08
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 22, 2017 3:36 AM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:
Am 21.07.2017 um 18:14 schrieb Enquiring Mind:

Hello,

it might be that the memory manager used in D7 programs is more prone to
memory fragmentation than the FastMM4 used since D2006.

You might want to download FastMM4 and test it. Use should be simple:
place the files in your source directory and add FastMM4 as first unit
in the uses of your .dpr file.
Thanks for the suggestion. 2-3 years ago I wrote a custom memory manager as an exercise - It worked and enabled me to visually see the structure of allocated memory in diagrammatic form, but I decided not to use it because I couldn't get it to work as fast as the default Delphi memory manager. So far, over many years of use, I have found the D7 memory manager satisfactory, so am a little wary of changing it just for the sake of a still unresolved bug or of slightly better performance.

EM
Roy Lambert

Posts: 959
Registered: 10/21/99
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 22, 2017 7:41 AM   in response to: Enquiring Mind in response to: Enquiring Mind
EM

FastMM is brilliant - just add it as the first unit in your project and it "just works" (now I'll get sued by Apple for using their advertising)

Roy Lambert

Markus Humm

Posts: 4,816
Registered: 11/9/03
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 23, 2017 5:24 AM   in response to: Roy Lambert in response to: Roy Lambert
Am 22.07.2017 um 16:41 schrieb Roy Lambert:
EM

FastMM is brilliant


At least for non multi threaded code afaik.

Greetings

Markus
Rudy Velthuis (...


Posts: 7,469
Registered: 9/22/99
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 21, 2017 2:28 PM   in response to: Enquiring Mind in response to: Enquiring Mind
Enquiring Mind wrote:

Hi,

An application I am currently developing with Delphi 7 has been
functioning satisfactorily for the last few weeks, but has recently
started intermittently throwing an "out of memory" exception at a
random point within the data loading procedure. That is to say, when
repeatedly running the same code to read the same data file from
disc, sometimes the procedure runs without error, and sometimes it
throws an "out of memory" exception when reading an object in a
collection of around 700 objects which can be d ifferent from time to
time. An example of a recent exception error message is:

Error reading TBookColn item ID 690 Token= "C" (Error in =
TBookBase.ReadFromTextStream for ID= 690 (Error in GetNextToken:
PrevToken= "," CurrToken= "C" CurrLine=", Subject: 184,
CategorySet: "[75, 144, 180]", Location: 49, BookType: "Soft cover",
], " (Out of memory)))

The data file being read has been getting bigger, and the memory
footprint of the global object has been also been increasing in size
as data is added to the data structure, but the size of each is still
quite low in relation to the capacity of the computer.

Then Alex is probably right by thinking of memory fragmentation. How do
you change the size of the buffer?

Also, read this:

http://rvelthuis.blogspot.de/2004/06/extending-arrays.html

Of course, his other points he makes also make sense. It could be a bug
in your code that only appears when certain things come together.

--
Rudy Velthuis http://www.rvelthuis.de

"When you are right you cannot be too radical; when you are
wrong you cannot be too conservative."
-- Martin Luther King, Jr.
Enquiring Mind

Posts: 87
Registered: 10/6/08
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 23, 2017 4:44 AM   in response to: Enquiring Mind in response to: Enquiring Mind
Enquiring Mind wrote:
Hi,

An application I am currently developing with Delphi 7 has been functioning satisfactorily for the last few weeks, but has recently started intermittently throwing an "out of memory" exception at a random point within the data loading procedure. That is to say, when repeatedly running the same code to read the same data file from disc, sometimes the procedure runs without error, and sometimes it throws an "out of memory" exception when reading an object in a collection of around 700 objects which can be different from time to time. An example of a recent exception error message is:

Error reading TBookColn item ID 690 Token= "C" (Error in = TBookBase.ReadFromTextStream for ID= 690 (Error in GetNextToken: PrevToken= "," CurrToken= "C" CurrLine=", Subject: 184, CategorySet: "[75, 144, 180]", Location: 49, BookType: "Soft cover", ], " (Out of memory)))

The data file being read has been getting bigger, and the memory footprint of the global object has been also been increasing in size as data is added to the data structure, but the size of each is still quite low in relation to the capacity of the computer.

Can anyone suggest what sort of problem could be causing these sporadic "out of memory" exceptions?
I think I have identified the procedure in which the exception is thrown:

procedure ReadAlphanumericalString(CurrToken: TStreamUTF32);
  {$define RecordPosn}
  {$undef RecordSteps}
  var
    SpecificAlphanumericalCharSet: TChar32Set;
    {$ifdef RecordPosn} PosnIndex: integer;   {$endif}
    {$ifdef RecordSteps} Steps: string;  {$endif}
  begin
    SpecificAlphanumericalCharSet:= TChar32Set.Create;
    try
      try
      {$ifdef RecordSteps} Steps:= '';  {$endif}
      if FAnsiAlphabeticalCharSet.Includes(FChar32) then
        begin
          {$ifdef RecordSteps} Steps:= 'A';  {$endif}
          {$ifdef RecordPosn} PosnIndex:= 0; {$endif}
          SpecificAlphanumericalCharSet.Assign(FAnsiAlphaNumericalCharSet);
          SpecificAlphanumericalCharSet.IncludeItem(ord('_'));
        end
      else if FNumericalCharSet.Includes(FChar32) then
        begin
          {$ifdef RecordSteps} Steps:= 'B'; {$endif}
          {$ifdef RecordPosn} PosnIndex:= 1; {$endif}
          SpecificAlphanumericalCharSet.Assign(FNumericalCharSet);
          SpecificAlphanumericalCharSet.IncludeItemArray([ord('e'), ord('E'), ord('+'),
             ord('-'), ord('.'), ord('_')]);
        end;
      {$ifdef RecordSteps} Steps:= Steps + 'C'; {$endif}
      {$ifdef RecordPosn} PosnIndex:= 2; {$endif}
      while SpecificAlphanumericalCharSet.Includes(FChar32) do       {While char is not a space or control char}
        begin
          CurrToken.WriteChar(FChar32);
          GetNextChar;
        end;
      {$ifdef RecordSteps} Steps:= Steps + 'D';  {$endif}
      {$ifdef RecordPosn} PosnIndex:= 3; {$endif}
      if CurrToken.CharCount<>0 then
        IdentifyAlphanumericalTokenType;
      {$ifdef RecordSteps} Steps:= Steps + 'E';  {$endif}
      {$ifdef RecordPosn} PosnIndex:= 4; {$endif}
      except
        on E: Exception do
          begin
            E.Message:= 'Error in ReadAlphanumericalString'
                {$ifdef RecordSteps}+ ' Steps= ' + Steps   {$endif}
                {$ifdef RecordPosn} + ' PosnIndex= ' + IntToStr(PosnIndex) {$endif}
                +  ' (' + E.Message + ')';
            raise;
          end;
      end;
    finally
      SpecificAlphanumericalCharSet.Free;
    end;
  end;
 

The original version of the above procedure did not include the statements enclosed in compiler directives, added later to assist in finding at what point the exception occurs. The interesting finding is that if {$define RecordSteps} is set, then an exception is NOT thrown; but if {$undef RecordSteps} is set, then an exception is thrown. The heap status at the time of the exception is:

TotalAddrSpace= 1938620416
TotalUncommitted= 1891991552
TotalAllocated= 2515920
TotalFree= 44044608
FreeSmall= 3291024
FreeBig= 40753584
Unused= 0
Overhead= 68336
HeapErrorCode= 0

The allocated memory doesn't look critical to me.

The message output by the exception handler is:

Stream size= 403713

(TBookColn.ReadFromTextStream Error in reading TBookBase ""Know the Game Squash"

(Error in = TBookBase.ReadFromTextStream for ID= 715

(Error in TUnicodeTextReaderStreamUnicodeBase.GetNextToken

(Error in ReadAlphanumericalString PosnIndex= 2 (Out of memory)))))

Any ideas why simply adding some code to record the steps traversed in the procedure should prevent the out of memory exception?

EM
Markus Humm

Posts: 4,816
Registered: 11/9/03
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 23, 2017 5:27 AM   in response to: Enquiring Mind in response to: Enquiring Mind
Hello,

I don't know in how far this checking code affects the issue,
but if you've already installed JEDI JCL, then you might want to
overwrite the global OnException handler in your main form and use the
stack tracing code from Indy to display the call stack when the
OutOfMemory exception occurs. Maybe it points to some even more specific
code line...

Greetings

Markus
Alex Belo

Posts: 535
Registered: 10/8/06
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 23, 2017 7:14 AM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:

to display the call stack when the OutOfMemory exception occurs

madExcept is also very helpful: it saves full callstack into log file.

--
Alex
Markus Humm

Posts: 4,816
Registered: 11/9/03
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 23, 2017 8:50 AM   in response to: Alex Belo in response to: Alex Belo
Am 23.07.2017 um 16:14 schrieb Alex Belo:
Markus Humm wrote:

to display the call stack when the OutOfMemory exception occurs

madExcept is also very helpful: it saves full callstack into log file.

Of course that's the option with more features.
It simply depends on whether he's developing a commercial application
and wants to speend the licence fee of that (which it well deserves!)

Greetings

Markus
Enquiring Mind

Posts: 87
Registered: 10/6/08
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 23, 2017 7:20 AM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:
Hello,

I don't know in how far this checking code affects the issue,
but if you've already installed JEDI JCL, then you might want to
overwrite the global OnException handler in your main form and use the
stack tracing code from Indy to display the call stack when the
OutOfMemory exception occurs. Maybe it points to some even more specific
code line...
Thank you for the useful suggestions in your posts. In this particular case I managed to trace the source of the exceptions without them, but I can see that they could be valuable for investigating exceptions that cannot be easily or quickly traced using the standard Delphi debugger.

EM
Olivier Sannier

Posts: 413
Registered: 8/26/01
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 7, 2017 11:53 PM   in response to: Markus Humm in response to: Markus Humm
Markus Humm wrote:
Hello,

I don't know in how far this checking code affects the issue,
but if you've already installed JEDI JCL, then you might want to
overwrite the global OnException handler in your main form and use the
stack tracing code from Indy to display the call stack when the
OutOfMemory exception occurs. Maybe it points to some even more specific
code line...

The JCL already contains stack tracing code, no need to add Indy on top
of it.

Markus Humm

Posts: 4,816
Registered: 11/9/03
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Aug 8, 2017 8:24 AM   in response to: Olivier Sannier in response to: Olivier Sannier
Am 08.08.2017 um 08:53 schrieb Olivier Sannier:
Markus Humm wrote:
Hello,

I don't know in how far this checking code affects the issue,
but if you've already installed JEDI JCL, then you might want to
overwrite the global OnException handler in your main form and use the
stack tracing code from Indy to display the call stack when the
OutOfMemory exception occurs. Maybe it points to some even more specific
code line...

The JCL already contains stack tracing code, no need to add Indy on top
of it.


You're right. Might be a typo of mine, Indy where It should have Jedi.

Greetings

Markus
Enquiring Mind

Posts: 87
Registered: 10/6/08
Re: Inconsistent throwing of exceptions  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 23, 2017 7:01 AM   in response to: Enquiring Mind in response to: Enquiring Mind
Enquiring Mind wrote:
I think I have identified the procedure in which the exception is thrown:

procedure ReadAlphanumericalString(CurrToken: TStreamUTF32);
  {$define RecordPosn}
  {$undef RecordSteps}
  var
    SpecificAlphanumericalCharSet: TChar32Set;
    {$ifdef RecordPosn} PosnIndex: integer;   {$endif}
    {$ifdef RecordSteps} Steps: string;  {$endif}
  begin
    SpecificAlphanumericalCharSet:= TChar32Set.Create;
    try
      try
      {$ifdef RecordSteps} Steps:= '';  {$endif}
      if FAnsiAlphabeticalCharSet.Includes(FChar32) then
        begin
          {$ifdef RecordSteps} Steps:= 'A';  {$endif}
          {$ifdef RecordPosn} PosnIndex:= 0; {$endif}
          SpecificAlphanumericalCharSet.Assign(FAnsiAlphaNumericalCharSet);
          SpecificAlphanumericalCharSet.IncludeItem(ord('_'));
        end
      else if FNumericalCharSet.Includes(FChar32) then
        begin
          {$ifdef RecordSteps} Steps:= 'B'; {$endif}
          {$ifdef RecordPosn} PosnIndex:= 1; {$endif}
          SpecificAlphanumericalCharSet.Assign(FNumericalCharSet);
          SpecificAlphanumericalCharSet.IncludeItemArray([ord('e'), ord('E'), ord('+'),
             ord('-'), ord('.'), ord('_')]);
        end;
      {$ifdef RecordSteps} Steps:= Steps + 'C'; {$endif}
      {$ifdef RecordPosn} PosnIndex:= 2; {$endif}
      while SpecificAlphanumericalCharSet.Includes(FChar32) do       {While char is not a space or control char}
        begin
          CurrToken.WriteChar(FChar32);
          GetNextChar;
        end;
      {$ifdef RecordSteps} Steps:= Steps + 'D';  {$endif}
      {$ifdef RecordPosn} PosnIndex:= 3; {$endif}
      if CurrToken.CharCount<>0 then
        IdentifyAlphanumericalTokenType;
      {$ifdef RecordSteps} Steps:= Steps + 'E';  {$endif}
      {$ifdef RecordPosn} PosnIndex:= 4; {$endif}
      except
        on E: Exception do
          begin
            E.Message:= 'Error in ReadAlphanumericalString'
                {$ifdef RecordSteps}+ ' Steps= ' + Steps   {$endif}
                {$ifdef RecordPosn} + ' PosnIndex= ' + IntToStr(PosnIndex) {$endif}
                +  ' (' + E.Message + ')';
            raise;
          end;
      end;
    finally
      SpecificAlphanumericalCharSet.Free;
    end;
  end;
 

Problem solved - it's an obscure programming error buried even deeper in the deep stack of nested procedure calls!
My previous post indicated that the error occurred after PosnIndex=2 and before PosnIndex= 3, i.e. in the while loop in which three procedures are called, namely SpecificAlphanumericalCharSet.Includes, CurrToken.WriteChar, and GetNextChar:

 {$ifdef RecordSteps} Steps:= Steps + 'C'; {$endif}
      {$ifdef RecordPosn} PosnIndex:= 2; {$endif}
      while SpecificAlphanumericalCharSet.Includes(FChar32) do       {While char is not a space or control char}
        begin
          CurrToken.WriteChar(FChar32);
          GetNextChar;
        end;
      {$ifdef RecordSteps} Steps:= Steps + 'D';  {$endif}
      {$ifdef RecordPosn} PosnIndex:= 3; {$endif}


Well, the error occurs in method GetNextChar, which reads:

procedure TUnicodeTextReaderBase.GetNextChar;
begin
  try
  if FCurrLine.EOF then      {If at end of current line}
    begin
      if FCharType= ctEOFChar then
        begin
        end
      else
        begin
          {Set next char to EOL marker}
          FChar32:= 13;
          FCharType:= ctSpaceChar;
          GetNextLine;
        end;
    end
  else
    begin
      {Get next char in current line}
      Inc(FCharIndex);
      FCurrLine.ReadChar(FChar32);
      IdentifyCharType;
     (* FStringBeforeCurrTag.WriteChar(FChar32); *)
    end;
  except
    on E: Exception do
      begin
        E.Message:= 'TUnicodeTextReaderBase.GetNextChar' + ' (' + E.Message + ')';
      end;
  end;
end;

The statement FStringBeforeCurrTag.WriteChar now commented out with the brackets (* .. *) appends the next char to the string FStringBeforeCurrTag. Since this string is a left-over from an earlier version in which the string had some significance, the method which clears this string is no longer called, with the result that this string never gets cleared and grows longer and longer, leading eventually to an out of memory condition. So commenting FStringBeforeCurrTag.WriteChar out resolves the out of memory problem.

To trace this problem I had to add try -- except statements to a large number of procedures where there weren't previously any.

Lesson to be learned: This case goes to show that when an out-of-memory exception is thrown, it is quite difficult to trace where it was thrown. Also, if a variable is no longer used, then it is best deleted rather than left for reuse at some future time.

EM

Edited by: Enquiring Mind on Jul 23, 2017 3:02 PM

Roy Lambert

Posts: 959
Registered: 10/21/99
Re: Inconsistent throwing of exceptions [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 23, 2017 7:31 AM   in response to: Enquiring Mind in response to: Enquiring Mind
EM

Congratulations on finding it. I loath intermittent bugs, give me a good old reproducible catastrophic crash any day.

Roy Lambert

Rudy Velthuis (...


Posts: 7,469
Registered: 9/22/99
Re: Inconsistent throwing of exceptions [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 24, 2017 2:38 AM   in response to: Roy Lambert in response to: Roy Lambert
Roy Lambert wrote:

Congratulations on finding it. I loath intermittent bugs, give me a
good old reproducible catastrophic crash any day.

I guess we can be glad you are not a bus driver. <g>

--
Rudy Velthuis http://www.rvelthuis.de

"Talk sense to a fool and he calls you foolish."
-- Euripides
Roy Lambert

Posts: 959
Registered: 10/21/99
Re: Inconsistent throwing of exceptions [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 24, 2017 4:52 AM   in response to: Rudy Velthuis (... in response to: Rudy Velthuis (...
Rudy

Congratulations on finding it. I loath intermittent bugs, give me a
good old reproducible catastrophic crash any day.

I guess we can be glad you are not a bus driver. <g>

Two points:

1, I'm not allowed anyway - I'm diabetic
2. You (as often the case) ignored the keyword - reproducible.

Roy
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02