Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: Big Data - Out of Memory


This question is not answered. Helpful answers available: 2. Correct answers available: 1.


Permlink Replies: 1 - Last Post: Jun 4, 2016 1:18 AM Last Post By: Dmitry Arefiev
Andrea Kaiser

Posts: 63
Registered: 11/17/15
Big Data - Out of Memory  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 1, 2016 2:46 PM
I have a big table and want to read it. At the statement 'Query.Open' I get the message "Out of memory". Basically it's the following routine:

    MyQuery.SQL.Text := 'select * from rhubarb';
    MyQuery.Open;
    while not MyQuery.Eof do begin
      (* get the record etc. *)
      MyQuery.Next;
    end;


I tried setting MyQuery.FetchOptions.Unidirectional to TRUE before the statement 'Open', but it doesn't help.

Apparently, at the command 'MyQuery.Open' all records are read and stored into the memory; that's why the memory overflows. Is there a possibility that only 1 record is physically read and at the statement 'MyQuery.Next' only the next record is read?
Processing time is not important!

Of course, I tried Google to find a solution, but I couldn't find one - so please be patient with me. :)
Has it something to do with FetchOptions fmOnDemand or RowsetSize?

Any help appreciated, particularly a little example would be great!

Edited by: Andrea Kaiser on Jun 1, 2016 2:47 PM

Edited by: Andrea Kaiser on Jun 1, 2016 2:47 PM
Dmitry Arefiev

Posts: 1,406
Registered: 12/7/03
Re: Big Data - Out of Memory  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 4, 2016 1:18 AM   in response to: Andrea Kaiser in response to: Andrea Kaiser
Please provide:
1) FireDAC environment report for your connection:
http://docwiki.embarcadero.com/RADStudio/Berlin/en/DBMS_Environment_Reports_%28FireDAC%29
2) "rhubarb" table DDL.

Anyway, try the following options:
1) Surround your code into FDConnection.StartTransaction / Commit
2) If you are using MySQL, then add connection parameter ResultMode=Use
3) If your table has large BLOB values, then exclude fiBlobs from MyQuery.FetchOptions.Items and Cache

--
With best regards,
Dmitry
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02