Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: [FireDAC][Phys][FB]-325. ArraySize [1] is less than ATimes [632]. Really?


This question is not answered. Helpful answers available: 2. Correct answers available: 1.


Permlink Replies: 5 - Last Post: Jan 14, 2016 1:11 PM Last Post By: Elias Sabbagh
Elias Sabbagh

Posts: 11
Registered: 2/28/00
[FireDAC][Phys][FB]-325. ArraySize [1] is less than ATimes [632]. Really?  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 12, 2016 2:27 PM
Hi folks!

I'm enjoying learning FireDAC, and I'm enjoying the speedups compared to dbExpress -- especially the Array DML feature. Unfortunately, I'm running into strange behavior with one routine that uses Array DML, and I'm at my wit's end (which is easy to reach, I'll admit).

I'm running some computations in Fortran, and when the F2003 code is done, it calls one or more Delphi callback routines to put various results into a Firebird 2.5 database. Each callback expects a pointer to some data item to be provided by the F2003 code. Callbacks use a parameterized SQL INSERT statement in a TFDQuery to store the records to the DB. I get the size of the data, which is usually a big array of some sort, from the callback parameters, and I want to use Array DML and set ArraySize of the parameters to the correct number of results from the Fortran routine. The callback uses the F2003 pointer to step through the raw data via pointer arithmetic, and to copy the values into the Params arrays. When the data is exhausted, we Execute() the Array DML. I've used this design in my app in a number of places, and once you get the parameter-passing conventions ironed out, everything works great. If you accidentally mess up either the parameter-passing conventions, or the pointer arithmetic, you can get some wild bugs. For instance, in other callback routines designed like this, some of these bugs really confused the Array DML code in FireDAC, overwriting the ArraySize property with garbage, for instance.

I've settled all of these problems except for one remaining callback. For the life of me, I can't figure out why I get the error message "[FireDAC][Phys][FB]-325. ArraySize [1] is less than ATimes [632]" for the following code, which is being tested with an ArraySize value of 632. There's no memory corruption as far as I can tell, and the parameter-passing conventions are correct. The count of records being stored via Firebird's EXECUTE BLOCK implementation of Array DML is not a problem, since we can't even get to the point of executing the Firebird statement, as verified via the FireDAC Monitor. I've edited the code shown below to remove a few other unrelated features such as logging -- and those have also been removed during testing to verify that they weren't the source of the bug, and the bug still occurs. What's left?

procedure PutLUDGMatPivot(<parameter that points to a data module where the various FDQueries live>, nun: integer; pludgmatpivot: Pointer); cdecl;
var
  pIntData: pInteger;
  ByprodID: Int64;
  i: cardinal;
  val: integer;
begin
  pIntData := pinteger(pludgmatpivot);
 
  // From the first parameter, we can access a data module that provides two FDQueries, named ByproductsFDQuery
  // and InsertFDQuery.  ByproductsFDQuery is a table of the various kinds of products that the F2003 code can make.
  // InsertFDQuery is a slave table to ByproductsFDQuery that provides the customized SQL INSERT statement
  // suitable for each kind of byproduct.
 
  // InsertFDQuery is rebuilt every time the master dataset ByproductsFDQuery's record changes.
  // In this particular test case, we've just previously used InsertFDQuery in a different callback to place a single
  // double-precision result in a different database table.  Is there a problem with reentrancy in the
  // implementation of Array DML?
 
  if( ByproductsFDQuery.Locate('BYPRODUCTTYPESHORTNAME', 'LUDGMatPivot', []) ) then
  begin
    // At this point, InsertFDQuery.SQL.Text is 'insert into vLUDGMatPivot (byproductid, i, pvt) values (:byproductid, :i, :pvt)'#$D#$A
    ByprodID := ...
    InsertFDQuery.Params.ArraySize := nun;     // the debugger reports nun = 632, the correct value for this test case
    for i := 0 to nun-1 do
    begin
       val := pIntData^;     // Everything checks out when I use the debugger to inspect the memory, both on the Delphi and F2003 sides
       InsertFDQuery.Params[0].AsLargeInts[i] := ByprodID;
       InsertFDQuery.Params[1].AsIntegers[i] := i;
       InsertFDQuery.Params[2].AsIntegers[i] := val;
       Inc(pIntData);
    end;
 
    if( nun > 0 ) then    // the debugger reports nun = 632, and InsertFDQuery.ArraySize = 632.  Everything's good.
       InsertFDQuery.Execute(nun);     // <-- BANG:  Exception says parameter [BYPRODUCTID] ArraySize [1] is less than ATimes [632].
 
       // Could the exception be due to the previous use of InsertFDQuery where a single double-precision value was placed into the DB using Array DML?
       // Is the Array DML code in FireDAC mistakenly holding onto the ArraySize of 1 from that previous use?
  end;
end;


I appreciate any tips for where to search next, or any workarounds that might come to mind. Thanks!
Elias Sabbagh

Posts: 11
Registered: 2/28/00
Re: [FireDAC][Phys][FB]-325. ArraySize [1] is less than ATimes [632]. Really?  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 12, 2016 10:08 PM   in response to: Elias Sabbagh in response to: Elias Sabbagh
Just determined some new information. I ran another test case that is similar to the first one, where a F2003 routine calls back to two different callbacks to store two arrays. The first callback succeeds without a hitch. The second callback, which is reusing the original InsertFDQuery object but with a changed INSERT statement, fails. My guess is that something's not being reset when the InsertFDQuery.SQL.Text property changes. This suggests obvious workarounds, which I'll try out, but it would be cleaner for me to keep the current design which reuses a singleton InsertFDQuery.
Dmitry Arefiev

Posts: 1,406
Registered: 12/7/03
Re: [FireDAC][Phys][FB]-325. ArraySize [1] is less than ATimes [632]. Really?  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 12, 2016 10:41 PM   in response to: Elias Sabbagh in response to: Elias Sabbagh
InsertFDQuery.Params.ArraySize := nun; // the debugger reports nun = 632, the correct value for this test case

Try the following:
* put breakpoint right after this line;
* run app up to this breakpoint;
* after stopping, press Ctrl+F4 (Evaluate/Modify);
* then evaluate values of:
** InsertFDQuery.Params.Count
** InsertFDQuery.Params[...].ArraySize

Does everything look correct ?

--
With best regards,
Dmitry
Elias Sabbagh

Posts: 11
Registered: 2/28/00
Re: [FireDAC][Phys][FB]-325. ArraySize [1] is less than ATimes [632]. Really?  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 13, 2016 10:24 AM   in response to: Dmitry Arefiev in response to: Dmitry Arefiev
Dmitry Arefiev wrote:
InsertFDQuery.Params.ArraySize := nun; // the debugger reports nun = 632, the correct value for this test case

Try the following:
* put breakpoint right after this line;
* run app up to this breakpoint;
* after stopping, press Ctrl+F4 (Evaluate/Modify);
* then evaluate values of:
** InsertFDQuery.Params.Count
** InsertFDQuery.Params[...].ArraySize

Does everything look correct ?

--
With best regards,
Dmitry

Hi Dmitry!

Yes, unfortunately everything looks correct -- all three params are present, and each has the same correct ArraySize property of 632. If I break right before calling Execute(), the same properties are still correct. I really suspect that reusing the TFDQuery is the problem. Simply changing the SQL.Text and changing ArraySize does not seem to clear out enough state from the last time Execute() was run. There's some sort of internal preparation that is hanging on.

The workaround would be to just dynamically create a fresh TFDQuery in each callback, of course.
Dmitry Arefiev

Posts: 1,406
Registered: 12/7/03
Re: [FireDAC][Phys][FB]-325. ArraySize [1] is less than ATimes [632]. Really?  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 13, 2016 8:43 PM   in response to: Elias Sabbagh in response to: Elias Sabbagh
Could you please provide a test application reproducing this issue.

--
With best regards,
Dmitry
Elias Sabbagh

Posts: 11
Registered: 2/28/00
Re: [FireDAC][Phys][FB]-325. ArraySize [1] is less than ATimes [632]. Really?  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 14, 2016 1:11 PM   in response to: Dmitry Arefiev in response to: Dmitry Arefiev
Dmitry Arefiev wrote:
Could you please provide a test application reproducing this issue.

--
With best regards,
Dmitry

We've applied a workaround -- simply create a new TFDQuery for each INSERT statement that will use Array DML -- so we're fine. I will try to create a test app to see if I can reproduce the behavior that occurs when there are multiple INSERTs consecutively handled by a singleton TFDQuery. More later...

Thanks for your help!
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02