Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: TApplication.OnIdle Event only fires when a dialog box is open


This question is answered. Helpful answers available: 2. Correct answers available: 1.


Permlink Replies: 31 - Last Post: Jan 28, 2015 3:49 PM Last Post By: Curtis Lending
Curtis Lending

Posts: 33
Registered: 11/24/14
TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 20, 2015 10:23 AM
I have inherited some code which was built with a older version of C++ Builder (XE5?) . It has background processing done using the TApplication.OnIdle Event.

When a dialog box is open, the OnIdle handler gets called repeatedly (which is what I want). If no dialog box is open, the handler is never called. I need the handler to be called regardless of the state of the program. Any suggestions?

Thanks,

Curtis
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 20, 2015 12:18 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

When a dialog box is open, the OnIdle handler gets called repeatedly
(which is what I want).

It should not be. The OnIdle event is triggered only when the main message
loop checks the message queue for a pending message and does not see one.
The message loop then waits for a new message to arrive, processes it, checks
for a new message, and if not found then triggers the OnIdle event and waits
for a new message. And so on. The sole purpose of the OnIdle event is to
let your code know when the message loop is entering an idling state when
no messages are being processed.

If you want something done periodically, use a TTimer.

If no dialog box is open, the handler is never called.

It is triggered whenever the message loop goes into an idle state when there
are no messages to process. If you are NEVER getting the OnIdle event triggered,
that can only mean that the message queue is never idling, it always has
a pending message waiting when the message loop checks it.

I need the handler to be called regardless of the state of the program.

No, you don't. That is not what the OnIdle event is meant for. You are
using the wrong solution to whatever problem you are trying to solve.

--
Remy Lebeau (TeamB)
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 20, 2015 1:35 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
I did not not design this code and it used to work properly. Apparently, switching to a new version of C++ Builder has caused the change in behavior.

It is triggered whenever the message loop goes into an idle state when there
are no messages to process. If you are NEVER getting the OnIdle event triggered,
that can only mean that the message queue is never idling, it always has
a pending message waiting when the message loop checks it.
If I do nothing, no Windows messages are received. If I do something (e.g. move the mouse), I get lots of appropriate messages. When I stop, the messages stop but I never get the OnIdle event.

If you want something done periodically, use a TTimer.
I have tried doing this with timers and it has failed for various reasons. I used the Windows SetTimer (which I believe TTimer is based on) and everything works but it is too slow because the smallest period in 10msec. Can I set an interval or 1 or 2 milliseconds for TTimer? I could not find this in the documentation.

If I used CreateTimerQueueTimer(), it is fast enough but parts of my graph do not appear. I know I am displaying it because if I minimize and re-open the window appears correctly. If I use Refresh or Update, the display gets messed up entirely.

Curtis
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 20, 2015 2:41 PM   in response to: Curtis Lending in response to: Curtis Lending
Another thing that seems odd to me. My application has drop down menus which open dialog boxes. As soon as click on a menu, WM_ENTERIDLE is received but the OnIdle callback is not hit. When I open a dialog box, I no longer see the WM_ENTERIDLE messages but the OnIdle procedure is hit.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 20, 2015 2:49 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

My application has drop down menus which open dialog boxes. As soon
as click on a menu, WM_ENTERIDLE is received but the OnIdle callback
is not hit.

The OnIdle event is not related to the WM_ENTERIDLE message.

When I open a dialog box, I no longer see the WM_ENTERIDLE messages
but the OnIdle procedure is hit.

Exactly what kind of dialog box are you displaying? This almost sounds like
the dialog box is running a non-VCL message loop that is swallowing messages.

--
Remy Lebeau (TeamB)
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 20, 2015 2:47 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

I did not not design this code and it used to work properly.
Apparently, switching to a new version of C++ Builder has caused
the change in behavior.

Based on your description, the only way the old code could have worked is
if the dialog in question was doing something that (directly or indirectly)
causing new messages to periodically arrive in the message queue, making
the message loop toggle from idle to active to idle to active and so on,
but the dialog in the new code is no longer doing that. What kind of dialog
is it? What kind of controls does it use? Maybe there was a UI change between
versions that affects messaging.

If I do nothing, no Windows messages are received.

Then the OnIdle event should be triggering almost immediately.

If I do something (e.g. move the mouse), I get lots of appropriate
messages.

Right. And when all of those messages have been processed, and there are
no more messages in the queue, the OnIdle event should trigger and the main
message loop will block until the next message arrives. Put a breakpoint
inside of TApplication.HandleMessage() to verify that.

TApplication.Run() calls TApplication.HandleMessage() in a loop until the
application is terminated. TApplication.HandleMessage() calls TApplication.ProcessMessage(),
which returns True or False to indicate whether a pending message was processed
or not. Once TApplication.ProcessMessage() returns False, TApplication.Idle()
is called, which triggers the TApplication.OnIdle event handler, performs
idle actions (TAction dispatching, TThread.Synchonize dispatching, etc) and
finally calls WaitMessage() to wait for a new message to appear in the message
queue, at which time flow goes back to TApplication.Run() and the loop repeats.

When I stop, the messages stop but I never get the OnIdle event.

That should only be possible if the dialog (or anything else) is actively
and regularly posting new messages to the message queue so TApplication never
goes idle.

I used the Windows SetTimer (which I believe TTimer is based on)

Correct.

and everything works but it is too slow because the smallest period in
10msec.

IIRC, the resolution of SetTimer() used to be like 50-60msec. Maybe it is
lower on newer OS versions?

Can I set an interval or 1 or 2 milliseconds for TTimer?

You can set it, and Windows will accept it, but will not actually honor it
because the timer resolution cannot go that low. Multimedia timers have
a better resolution than SetTimer(), but certainly not at 1ms intervals.
Windows simply does not run that fast.

If I used CreateTimerQueueTimer(), it is fast enough but parts of my
graph do not appear. I know I am displaying it because if I minimize
and re-open the window appears correctly. If I use Refresh or Update,
the display gets messed up entirely.

That suggests to me that either you are not drawing the graph correctly to
being with, or more likely the timer callback is running in a different thread
context than you are expecting. Remember that UI access and UI drawing has
to be done in the context of the main UI thread. TTimer would run in the
main thread, if created in the main thread. CreateTimerQueueTimer() does
not offer that same quarantee.

--
Remy Lebeau (TeamB)
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 21, 2015 10:03 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
more likely the timer callback is running in a different thread
context than you are expecting. Remember that UI access and UI drawing has
to be done in the context of the main UI thread. TTimer would run in the
main thread, if created in the main thread. CreateTimerQueueTimer() does
not offer that same quarantee.
You are correct that the timer callback is in a different thread. Do you know how to solve this problem? The flag that seems to make sense, WT_EXECUTEINIOTHREAD, is "not used" according to the Microsoft documentation.

Thanks for you help,

Curtis
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 21, 2015 12:40 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

You are correct that the timer callback is in a different thread.
Do you know how to solve this problem?

Without seeing your actual code, the only thing I can suggest is make sure
your timer callback code is thread-safe, and any direct UI access must be
delegated to the main UI thread. Any drawing that the timer wants to do
can be done to an in-memory bitmap which the main thread can then draw onscreen
when needed.

--
Remy Lebeau (TeamB)
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 21, 2015 1:23 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
I tried something else that seems like it should work but did not work either.

In my timer callback, I tried to post a message to the main thread using:
	err = PostThreadMessage(threadId,WM_USER, 5, 5);
	err = PostMessage(Application->MainFormHandle,WM_USER, 5, 5);

The err code is 1 indicating success.

I expected to receive this in my OnMessage handler but it was not triggered. I know the handler is functional because it is hit when I have a dialog box open. BTW, you were correct OnIdle not firing because no messages are received empty during normal operation (i.e. the queue is always empty).

Thanks,

Curtis
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 21, 2015 2:10 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

In my timer callback, I tried to post a message to the main thread
using:
{code}
err = PostThreadMessage(threadId,WM_USER, 5, 5);

That should be fine, provided threadid is the same value as the global MainThreadID
variable (which you could just use directly). The TApplication message loop
supports thread messages.

err = PostMessage(Application->MainFormHandle,WM_USER, 5, 5);

You have to be careful with that. The MainFormHandle property may end up
reading the MainForm->Handle property, but the TWinControl::Handle property
is not thread-safe.

I expected to receive this in my OnMessage handler

Under normal conditions, yes you should.

but it was not triggered.

The only way that should be happening is if the main thread message loop
is not receiving the message, such as if there is a modal OS dialog running
its own message loop that is swallowing messages. Raymond Chen warned about
that several times in his "Old New Thing" blog on MSDN:

Thread messages are eaten by modal loops
http://blogs.msdn.com/b/oldnewthing/archive/2005/04/26/412116.aspx

Watching thread messages disappear
http://blogs.msdn.com/b/oldnewthing/archive/2005/04/27/412565.aspx

Rescuing thread messages from modal loops via message filters
http://blogs.msdn.com/b/oldnewthing/archive/2005/04/28/412574.aspx

Why do messages posted by PostThreadMessage disappear?
http://blogs.msdn.com/b/oldnewthing/archive/2009/09/30/9901065.aspx

I know the handler is functional because it is hit when I have a dialog box
open. BTW, you were correct OnIdle not firing because no messages are
received empty during normal operation (i.e. the queue is always empty).

That goes back to suggesting the message(s) are being swallowed before the
VCL sees them.

--
Remy Lebeau (TeamB)
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 21, 2015 2:35 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
The only way that should be happening is if the main thread message loop
is not receiving the message, such as if there is a modal OS dialog running
its own message loop that is swallowing messages. Raymond Chen warned about
that several times in his "Old New Thing" blog on MSDN:
If something is swallowing messages, would I still get an OnIdle event?

I added calls to Application->HandleMessage(). This triggered the OnMessage handler which kind of makes sense but the message numbers did not 2304, 799, 49347 and lots of 512. But the OnIdle did not trigger.

I guess I am still confused by the OnIdle handler. If OnMessage gets triggered, then I would expect OnIdle to be triggered immediately afterwards because the queue becomes empty.

Curtis
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 21, 2015 7:10 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

If something is swallowing messages, would I still get an OnIdle
event?

Yes, if flow returns to the TApplication message loop and it then detects
an empty message queue. But if something else is running its own message
loop without using TApplication.HandleMessage() to process messages than
all bets are off.

I added calls to Application->HandleMessage().

Where?

This triggered the OnMessage handler which kind of makes sense
but the message numbers did not 2304, 799, 49347 and lots of 512.

2304 is 0x0900, which is WM_USER + 1280, aka a message used by a private
window class. System-defined window class messages do not tend to go that
high, so it is likely a private message used by a 3rd party UI control/library.

799 is 0x031F, which is a reserved system message.

49347 is 0xC0C3, which is message registered via RegisterWindowMessage()
at runtime.

512 is 0x0200, which is a reserved system message.

But the OnIdle did not trigger.

Then the message queue likely did not empty out.

I guess I am still confused by the OnIdle handler.

I explained to you how it works. But it only works if TApplication is managing
the message loop.

If OnMessage gets triggered, then I would expect OnIdle to be triggered
immediately afterwards because the queue becomes empty.

The OnMessage event being triggered does not guarantee the message queue
is empty. OnMessage is triggered whenever TApplication pulls a message off
the top of the queue. Other messages can still exist in the queue, or be
put into the queue while the pulled message is being processed.

--
Remy Lebeau (TeamB)
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 22, 2015 1:10 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy,

I appreciate you help and I feel I am making progress but it is not quite working.

The appropriate solution to seems to be that I use CreateTimerQueueTimer() to trigger a callback in another thread. The timer thread can then send a message via PostThreadMessage() to the main thread which can do the processing and update the display.

The problem I am having is that the line:
err = PostThreadMessage(MainThreadID ,WM_USER+10 ,0,0);

returns success (1) but the OnMessage callback does not get called.. If something else was intercepting the message, I would expect Onldle to be called. I know both of these callbacks are working because they are triggered when I open a dialog box.

It seems to me that PostThreadMessage() is sending elsewhere or not really succeeding. Do have any suggestions on how I could figure out what is failing?

This might be a clue. In trying to diagnose the problem, instead of the timer, I added the line
Application->HandleMessage();

to the main thread. The caused OnMessage to fire with a 2304 (0x900) message which makes sense. But the OnIdle does not get triggered which does not.

<OnIdle> only works if TApplication is managing the message loop. ….
...
The OnMessage event being triggered does not guarantee the message queue
is empty. OnMessage is triggered whenever TApplication pulls a message off
the top of the queue. Other messages can still exist in the queue, or be
put into the queue while the pulled message is being processed.

What else could be managing the message loop? How would this been done? How would I know if it occurring?

Curtis
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 22, 2015 1:28 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

The appropriate solution to seems to be that I use CreateTimerQueueTimer()
to trigger a callback in another thread. The timer thread can then send a
message via PostThreadMessage() to the main thread which can do the
processing and update the display.

Yes.

The problem I am having is that the line:

err = PostThreadMessage(MainThreadID ,WM_USER+10 ,0,0);


returns success (1) but the OnMessage callback does not get called..

As I explained earlier, there is only one way that can happen - something
OTHER THAN TApplication is receiving the message.

An alternative approach would be to use PostMessage() instead of PostThreadMessage()
(like Raymond Chen suggested in his blog). You can use the Application->Handle
window to post messages to, and then use the Application->OnMessage or TApplicationEvents::OnMessage
event to receive them. Or create a dedicated window using AllocateHWnd()
instead.

If something else was intercepting the message, I would expect Onldle
to be called.

Only if Application->HandleMessage() sees an empty message queue.

I know both of these callbacks are working because they are triggered
when I open a dialog box.

That means absolutely nothing to me without seeing what you are seeing and
doing.

It seems to me that PostThreadMessage() is sending elsewhere or not
really succeeding.

Not true on both counts.

This might be a clue. In trying to diagnose the problem, instead of
the timer, I added the line

Application->HandleMessage();


to the main thread.

But WHERE did you add it? What is triggering the call?

The caused OnMessage to fire with a 2304 (0x900) message which
makes sense. But the OnIdle does not get triggered which does not.

Put a breakpoint on the HandleMessage() call and step into it with the debugger
(turn on Debug DCUs in the project options). Keep stepping and you will
see why OnIdle is not being triggered. You are likely to find that the message
queue always has a message waiting when you are calling HandleMessage().
That is the only way OnIdle would not be triggered. So something in the
main thread is pumping new messages into the queue, preventing the app from
going idle.

What else could be managing the message loop?

Modal dialogs, menu navigation, etc. Things that require processing new
UI messages while blocking your code.

How would this been done? How would I know if it occurring?

I cannot answe that without seeing some real code showing what you are actually
doing. It is very difficult to diagnose your problem when we canot even
see what you are doing.

--
Remy Lebeau (TeamB)
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 23, 2015 11:12 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy,

I understand that idle is only triggered when the message queue goes from not empty to empty. I understand that messages may be intercepted.

Here are things that I would assume:
1) Messages should either be intercepted or go to the OnMessage callback.
2) If a message is intercepted, it should be removed from the queue. Is this correct?
3) If a message is handled by the OnMessage handler, it should be removed from the queue.
4) If 1, 2 and 3 are true, then if I post a message,
a. It will go on the queue.
b. It should be removed from the queue by something.
c. The queue should go from not empty to empty.
d. The OnIdle should be triggered.
But I am not getting the OnIdle even though the message appears to be sent correctly. What I am missing?

I have read the posts by Raymond Chen and I have tried using:
PostMessage instead of PostThreadMessage
Using WM_NULL to try to get the queue to go empty and trigger the OnIdle.

In the code I am trying to get work I am not using HandleMessage. I am expecting either OnIdle or OnMessage to be triggered. I was using it only to try to understand what was going on in the queue. I am trying to debug the HandleMessage as you suggested but it does not seem to be working. I am still looking into this.

In the main thread, when the user clicks start, I have the code
		tResult = CreateTimerQueueTimer( &hTimer, TIMER_QUEUE,
				(WAITORTIMERCALLBACK)TimerHandler, &threadId , TIMER_LENGTH, 0, WT_EXECUTEINIOTHREAD);


Here is the timer handler
void CALLBACK TimerHandler(HWND hwnd, UINT uMsg, UINT timerId, DWORD dwTime)
 
{
	DBG_PRINTF("TimerHandler");
	#ifdef POST_MSG
		DWORD err;
		UINT msg;
		msg = WM_USER;
		err = PostThreadMessage(MainThreadID,msg, 5, 5);
		DBG_PRINTF("PostThreadMessage %d to %d, err: %d %d", msg, MainThreadID, err,GetLastError());
 
		msg = WM_NULL;
		err = PostThreadMessage(MainThreadID ,msg ,0,0);
		DBG_PRINTF("PostThreadMessage %d to %d, err: %d %d", msg, MainThreadID, err,GetLastError());
 
		msg = WM_USER;
		err = PostMessage(Application->MainFormHandle,msg, 5, 5);
		DBG_PRINTF("PostMessage %d, err: %d %d", msg, err,GetLastError());
 
		msg = WM_NULL;
		err = PostMessage(Application->MainFormHandle,msg, 5, 5);
		DBG_PRINTF("PostMessage %d, err: %d %d", msg, err,GetLastError());
 
		err = SendMessage(Application->MainFormHandle,msg, 5, 5);
		DBG_PRINTF("SendMessage %d, err: %d %d", msg, err,GetLastError());
 
	#else
		bool Done;
		dummyThis->windowsIdle(NULL, Done);
	#endif
 
}


PostThreadMessage and PostMessage return an error value of 1. SendMessage returns an error value of zero.

Message handler:
DLLEXPORT void __fastcall SYSTEM::OnMessageHandler(tagMSG &Msg, bool &Handled)
{
 
	DBG_PRINTF("+++++++++++++++OnMessage %d (0x%x)", Msg.message, Msg.message);
}


Idle Handler
DLLEXPORT void __fastcall SYSTEM::windowsIdle(TObject *Sender, bool &Done)
{
	DBG_PRINTF("--windowsIdle - Done %d", Done);
idleHook->activate(1, &Done);
}


Thank you,

Curtis
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 23, 2015 1:46 PM   in response to: Curtis Lending in response to: Curtis Lending
Remy,

With your help, I almost have it working. If I call HandleMessage multiple times before starting the timer, then both OnMessage and OnIdle get triggered. I thought that OnMessage would be triggered when an message went in the queue and did not realize that calling HandleMessage was necessary. This seems odd to me (and a co-worker) but as long as I can make it work, odd is OK.

I am not sure when and how many time I need to call HandleMessage because I still do not understand the exact relationship between HandleMessage, the OnMessage handler and WndProc. I am looking into this but if you have a link, that would be great.

Thanks again,

Curtis
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 26, 2015 12:45 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

If I call HandleMessage multiple times before starting the timer, then both
OnMessage and OnIdle get triggered. I thought that OnMessage would be
triggered when an message went in the queue and did not realize that
calling HandleMessage was necessary.

I told you 6 days ago that HandleMessage() was responsible for that:

TApplication.Run() calls TApplication.HandleMessage() in a loop until the
application is terminated. TApplication.HandleMessage() calls TApplication.ProcessMessage(),
which returns True or False to indicate whether a pending message was processed
or not. Once TApplication.ProcessMessage() returns False, TApplication.Idle()
is called, which triggers the TApplication.OnIdle event handler, performs
idle actions (TAction dispatching, TThread.Synchonize dispatching, etc) and
finally calls WaitMessage() to wait for a new message to appear in the message
queue, at which time flow goes back to TApplication.Run() and the loop repeats.

TApplication.ProcessMessage() triggers OnMessage if a message is retreived
from the queue. TApplication.Idle() triggers OnIdle.

This seems odd to me (and a co-worker) but as long as I can make it work,
odd is OK.

You should NOT be calling HandleMessage() directly. Let the VCL call it
internally as needed. If OnMessage and OnIdle are not triggering correctly,
then you need to track down what is interferring with your message queue.
SOMETHING in your project is dipping into it.

--
Remy Lebeau (TeamB)
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 26, 2015 12:36 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

1) Messages should either be intercepted or go to the OnMessage callback.

If a message is posted to the queue, but is not seen in the TApplication.OnMessage
event, then TApplication is not the one retreiving the message from the queue.

2) If a message is intercepted, it should be removed from the queue. Is
this correct?

That depends on how it was retreived from the queue.

3) If a message is handled by the OnMessage handler, it should be
removed from the queue.

Yes.

4) If 1, 2 and 3 are true, then if I post a message,
a. It will go on the queue.
b. It should be removed from the queue by something.
c. The queue should go from not empty to empty.
d. The OnIdle should be triggered.

Again, as I keep telling you, ONLY IF TApplication IS THE ONE PERFORMING
MESSAGE RETREIVAL! The fact that you are having so many problems getting
the OnIdle event to trigger must mean you have something else messing around
with the message behind TApplication's back.

err = PostMessage(Application->MainFormHandle,msg, 5, 5);

Application->MainFormHandle is not thread-safe. I told you that before.
Use Application->Handle oor AllocateHwnd() instead.

PostThreadMessage and PostMessage return an error value of 1.
SendMessage returns an error value of zero.

Messages sent via SendMessage() do not go through the message queue, and
thus will not trigger the OnMessage event.

--
Remy Lebeau (TeamB)
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 27, 2015 6:56 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
In the timer callback, I have the following line:
	msg = WM_USER;
	err = PostMessage(Application->Handle,msg, 5, 5);


If everything is working properly, should this message automatically go to the OnMessage handler? Or do I need to do something in the main thread?

If I need to do something, what do I need to add to the main thread?

If the main thread does not need to something, how do I diagnose what is going wrong? Calling HandleMessage repeatedly will make the OnMessage handler receive some other messages (0x900,0x31f,0xc0d0,0x2a3,0xf,0xf,0xf,0xf,0xf,0xf,0xf) and then the OnIdle will trigger. Then I get the WM_TIMER (0x113) message alternating with the WM_USER message.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 27, 2015 8:19 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

In the timer callback, I have the following line:

msg = WM_USER;
err = PostMessage(Application->Handle,msg, 5, 5);


If everything is working properly, should this message automatically
go to the OnMessage handler?

Yes.

--
Remy Lebeau (TeamB)
david hoke

Posts: 616
Registered: 2/9/07
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 28, 2015 8:15 AM   in response to: Curtis Lending in response to: Curtis Lending
<Curtis Lending> wrote in message news:711832 at forums dot embarcadero dot com...
If the main thread does not need to something, how do I diagnose what is
going wrong? Calling HandleMessage repeatedly will make the OnMessage
handler receive some other messages
(0x900,0x31f,0xc0d0,0x2a3,0xf,0xf,0xf,0xf,0xf,0xf,0xf) and then the OnIdle
will trigger. Then I get the WM_TIMER (0x113) message alternating with
the WM_USER message.

Possibly, find a tracer of some sort that will log callstacks, and see
who/what is calling GetMessage()/PeekMessage() (I'm guessing, that
GetMessage() may itself call PeekMessage...), and log the callstacks of
callers and see if there is a caller that you aren't aware of...

You might be able to use one of either gdb, or ollydbg
(http://www.ollydbg.de/), for this purpose. (Note that I have no experience
with ollydbg, so might not be suitable.)

There is a gdb hack that has some awareness of tds debug info format at
gdbwtds.sf.net.

While I haven't ever used the functionality, I believe gdb has the
possibility of setting 'print' outputs at break/watch-points and continuing
on, so you might be able to set up a watchpoint on entry to PeekMessage()
(and ?), and request the stack info ('info stack'), before continuing on...

Maybe a breakpoint with the following functionality would work best...
https://sourceware.org/gdb/onlinedocs/gdb/Break-Commands.html
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 28, 2015 9:42 AM   in response to: david hoke in response to: david hoke
david wrote:

Possibly, find a tracer of some sort that will log callstacks, and
see who/what is calling GetMessage()/PeekMessage()

Or, just have your own app use a detour to hook into GetMessage() itself,
and then use the IDE debugger to put a breakpoint inside your detour and
look at the call stack directly.

--
Remy Lebeau (TeamB)
david hoke

Posts: 616
Registered: 2/9/07
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 28, 2015 10:31 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
"Remy Lebeau (TeamB)" <no dot spam at no dot spam dot com> wrote in message
news:711939 at forums dot embarcadero dot com...
david wrote:

Possibly, find a tracer of some sort that will log callstacks, and
see who/what is calling GetMessage()/PeekMessage()

Or, just have your own app use a detour to hook into GetMessage() itself,
and then use the IDE debugger to put a breakpoint inside your detour and
look at the call stack directly.

Remy's suggestion is also viable, maybe even better/easier (I assume he's
referring to use of the detours [I think its called] library, possibly from
MS), but...

1)Don't forget about PeekMessage() possibility, I think it can also remove
from queue...

2)Don't ignore possibility of logging, as the number of messages flowing
through the Q can be large, and I at least tire of inspecting/continuing at
each breakpoint (loss of concentration) with something that is lengthy (many
repetitions), finding it a bit easier often with a large amount of data to
visually scan logged stuff looking for items that are
unexpected/out-of-place, so, rather than breakpointing at your detour, just
log call stack at the detour and continue.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 28, 2015 10:54 AM   in response to: david hoke in response to: david hoke
david wrote:

Remy's suggestion is also viable, maybe even better/easier (I assume
he's referring to use of the detours [I think its called] library,
possibly from MS)

There a plenty of API hooking libraries available.

1)Don't forget about PeekMessage() possibility, I think it can also
remove from queue...

Yes.

2)Don't ignore possibility of logging, as the number of messages
flowing through the Q can be large, and I at least tire of
inspecting/continuing at each breakpoint (loss of concentration) with
something that is lengthy (many repetitions), finding it a bit easier
often with a large amount of data to visually scan logged stuff
looking for items that are unexpected/out-of-place, so, rather than
breakpointing at your detour, just log call stack at the detour and
continue.

Or, simply configure the breakpoint to only break when a specific message
has been removed from the queue, preferrably one that is not being dispatched
correctly.

--
Remy Lebeau (TeamB)
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 28, 2015 10:57 AM   in response to: david hoke in response to: david hoke
david wrote:

Remy's suggestion is also viable, maybe even better/easier (I assume
he's referring to use of the detours [I think its called] library,
possibly from MS)

There a plenty of API hooking libraries available.

1)Don't forget about PeekMessage() possibility, I think it can also
remove from queue...

Yes.

2)Don't ignore possibility of logging, as the number of messages
flowing through the Q can be large, and I at least tire of
inspecting/continuing at each breakpoint (loss of concentration) with
something that is lengthy (many repetitions), finding it a bit easier
often with a large amount of data to visually scan logged stuff
looking for items that are unexpected/out-of-place, so, rather than
breakpointing at your detour, just log call stack at the detour and
continue.

Or, simply configure the breakpoint to only break when a specific message
has been removed from the queue, preferrably one that is not being dispatched
correctly.

--
Remy Lebeau (TeamB)
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 28, 2015 12:00 PM   in response to: david hoke in response to: david hoke
david wrote:

Remy's suggestion is also viable, maybe even better/easier (I assume
he's referring to use of the detours [I think its called] library,
possibly from MS)

There a plenty of API hooking libraries available.

1)Don't forget about PeekMessage() possibility, I think it can also
remove from queue...

Yes.

2)Don't ignore possibility of logging, as the number of messages
flowing through the Q can be large, and I at least tire of
inspecting/continuing at each breakpoint (loss of concentration) with
something that is lengthy (many repetitions), finding it a bit easier
often with a large amount of data to visually scan logged stuff
looking for items that are unexpected/out-of-place, so, rather than
breakpointing at your detour, just log call stack at the detour and
continue.

Or, simply configure the breakpoint to only break when a specific message
has been removed from the queue, preferrably one that is not being dispatched
correctly.

--
Remy Lebeau (TeamB)
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 28, 2015 11:02 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Or, just have your own app use a detour to hook into GetMessage() itself,
and then use the IDE debugger to put a breakpoint inside your detour and
look at the call stack directly.
I am unfamiliar with detours. Are they a separate library? Where would I find them?

I have two other questions:
1) I cannot step into HandleMessage even though I have set "Options->Delphi Compiler->Compiling-Use debug .dcus" to true. Is there something else I need to set in options?
2) If I call HandleMessage repeatedly from my main thread I think can get this to work. I know when I need to stop processing so I will only do this when I am expecting a message. I realize this is not the best solution but is there a problem with this? I am under pressure to get something working.

Thanks,

Curtis
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 28, 2015 12:14 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

I am unfamiliar with detours.

Detours: Binary Interception of Win32 Functions
http://research.microsoft.com/apps/pubs/default.aspx?id=68568

Are they a separate library?

There are 3rd party libraries that implement detouring, yes. Such as:

https://code.google.com/p/delphi-detours-library/wiki/DDetours

1) I cannot step into HandleMessage even though I have set
"Options->Delphi Compiler->Compiling-Use debug .dcus" to true.
Is there something else I need to set in options?

Are you compiling with Runtime Packages and Dynamic RTL both disabled? Do
you have the VCL source folder in your project's source path?

2) If I call HandleMessage repeatedly from my main thread I think can
get this to work.

TApplication already does that. The fact that you have to do it manually
tells you that something is wrong with your project. You are doing something
you should not be doing, or you are relying on functionality that is not
reliable.

I know when I need to stop processing so I will only do this when I am
expecting a message. I realize this is not the best solution but is there
a problem with this? I am under pressure to get something working.

Switching to PostMessage() or SendMessage() should work fine, no matter who
is pumping the message queue. Messages still have to be dispatched, but
at least HWND message dispatching is more reliable and predictable than thread
message dispatching. That is what Raymond Chen's blog tried to explain.
Just make sure to use an HWND that will not disappear on you unexpectedly
(IOW, TApplication->MainFormHandle, MainForm->Handle, any TWinControl->Handle,
etc). I strongly suggest using AllocateHWnd() for custom messages.

--
Remy Lebeau (TeamB)
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 28, 2015 1:29 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Switching to PostMessage() or SendMessage() should work fine, no matter who
is pumping the message queue.
My timer callback says:
err = PostMessage(Application->Handle,msg, 5, 5);

but I do not receive messages unless I do the HandleMessage() in the main thread. If I understand you, I should be receiving it regardless of what is happening elsewhere..

Are you compiling with Runtime Packages and Dynamic RTL both disabled? Do
you have the VCL source folder in your project's source path?
I was linking with run time packages but I when I turn that I off, I get the message:
Project Ishmael.exe raised exception class EConnvertError with message 'Cannot assign a TFont to a TFont'.

from the code:
procedure TControl.SetFont(Value: TFont);
begin
  FFont.Assign(Value);
end;


Do I need to get this working to use the detours?

Thanks,

Curtis
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 28, 2015 1:48 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

My timer callback says:

err = PostMessage(Application->Handle,msg, 5, 5);


but I do not receive messages unless I do the HandleMessage()
in the main thread.

Right, because that is the only way to get a queued message delivered to
the TApplication::OnMessage event. But that event is not the only way to
receive a message that is destined for the TApplication window. You can
also use the Application->HookMainWindow() method. The difference is that
OnMessage is triggered for any message that TApplication polls from the queue,
whereas HookMainWindow() creates a hook that the TApplication window procedure
calls for any message that the Tapplication window receives whether it came
from the message queue or not.

Again, if you are posting messages to yourself, you should create your own
HWND that you control, and then post to that HWND. That is exactly what
AllocateHWnd() is for. Did you even try it yet?

I was linking with run time packages

You cannot step into the VCL source code when Runtime Packages are enabled.

but I when I turn that I off, I get the message:
Project Ishmael.exe raised exception class EConnvertError with message
'Cannot assign a TFont to a TFont'.

That happens when a TFont object is being Assign()'ed to another TForm object
across DLL boundaries. The RTL, and thus RTTI, inside the DLL would be different
than the RTL/RTTI inside the EXE, and thus Assign() would fail to recognize
a DLL-created TFont object and an EXE-created TFont object as belonging to
the same class. When Runtime Packages are enabled, both EXE and DLL share
a common RTL and RTTI, so there is no failure.

--
Remy Lebeau (TeamB)
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 28, 2015 2:13 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy,

I need to get something working this week even if it is a little buggy. For now, I am going to have my code use the HandleMessage() in the main thread.

Sometime, I will come back to this and make it work properly. It may be this week if things go well or a few months but it's going on the back burner for now.

Thanks for your help,

Curtis
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TApplication.OnIdle Event only fires when a dialog box is open  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jan 28, 2015 3:49 PM   in response to: Curtis Lending in response to: Curtis Lending
Remy,

I have solved my initial problem of the OnIdle event not being triggered by changing build options which means I can forget about dealing with the queue. It also means the code is running as designed which makes me feel better.

I had been compiling one of 30 subprojects with RELEASE (because DEBUG caused a link error) and the other 29 with DEBUG. I solved the problem in the one subproject and now that all the subprojects are compiling successfully with DEBUG, the idles occur all the time.

Thanks for you help. I learned a lot about timers, queues and idle even though it ultimately had nothing to with my root issue.

Curtis
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02