Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: TIdTCPClient lags and times out


This question is answered. Helpful answers available: 2. Correct answers available: 1.


Permlink Replies: 5 - Last Post: Jun 30, 2015 2:36 PM Last Post By: Curtis Lending
Curtis Lending

Posts: 33
Registered: 11/24/14
TIdTCPClient lags and times out  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 16, 2015 2:23 PM
I have a project where I receive data from several external devices using a TiDTCPClient. In one situation, the external devices spew data faster than my program can read and process. When I fail to call IOHandler->ReadBytes() often enough, this causes the TCP sockets to timeout and disconnect.

When I cannot read data fast enough, I would like the TCP data to be stored in an extremely large buffer. Later when the flow slows, I would gradually read the data out of buffer. Is there a way to accomplish this? My code to read the data is:

TIdBytes readBuffer;
ChanTCPClient[chIndex]->IOHandler->ReadBytes(readBuffer, 576, APPEND);

Alternately, I could using an interrupt like thing to read the data asynchronously but I did not see a way to do this either. I thought the Intercept stuff might be helpful but the documentation says it "is used to perform operations that result from connecting to or disconnecting" not reading.

In my online search, I saw stuff about TidCmdTcpClient but this is not in the C++ builder help.

Thank you,

Curtis
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdTCPClient lags and times out  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 16, 2015 8:20 PM   in response to: Curtis Lending in response to: Curtis Lending
Curtis wrote:

I have a project where I receive data from several external devices
using a TiDTCPClient. In one situation, the external devices spew
data faster than my program can read and process. When I fail to
call IOHandler->ReadBytes() often enough, this causes the TCP
sockets to timeout and disconnect.

What does your reading code actually look like? How large are the TCP messages?
How fast are they actually being sent? How many devices are you connected
to at one time?

When I cannot read data fast enough, I would like the TCP data to
be stored in an extremely large buffer. Later when the flow slows,
I would gradually read the data out of buffer. Is there a way to
accomplish this?

Indy already uses a 32K receive buffer by default, but you can always set
it higher (see the TIdIOHandler.RecvBufferSize property).

My code to read the data is:

TIdBytes readBuffer;
ChanTCPClient[chIndex]->IOHandler->ReadBytes(readBuffer, 576, APPEND);

But how often are you actually calling it? And are you calling it in the
main thread, or in a worker thread? The fact that you have a client array
makes me think you are looping through the clients sequentially. Are you?
Or does each client run in its own thread?

--
Remy Lebeau (TeamB)
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TIdTCPClient lags and times out  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 17, 2015 2:18 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...

What does your reading code actually look like? How large are the TCP messages?
How fast are they actually being sent? How many devices are you connected
to at one time?
Each device (up to 6) sends 333 packets of length 576 (192K bytes per channel or total of 1.1MByte) per second. The sending devices send data without any application level response required i.e. they only wait for the TCP Ack. I can handle this fine under normal processing but there are times when the rest of the system does intense mathematical processing and I do not call the ReadBytes fast enough.

I call the follow routine in a loop
bool __fastcall TM3RParamsForm::ReadChannel(long channel, M3R_DATA_PACKET *packet, U16 *seqNum) {
	int chIndex = getIndexForChannel(channel);
 
	// Assume good packet
	bool retVal = true;
 
	*seqNum = 0xffff;
 
	// Read the data
	TIdBytes readBuffer;
 
	#define APPEND			false
		try {
			ChanTCPClient[chIndex]->IOHandler->ReadBytes(readBuffer, M3R_EXPECTED_LENGTH, APPEND);	
			if (readBuffer.Length <= 0) {
				DBG_PRINT("Stop - read done");
				retVal = false;
			}
			if (readBuffer.Length == M3R_EXPECTED_LENGTH ) {
				if (MoveAndCheckPacket(readBuffer, packet)) {
					*seqNum = packet->header.hdr.seq_num;
				}
				else {
					retVal = false;
				}
			}
		}
		catch (...) {
			gError(STR_M3R_COMM_FAIL);
			DisconnectChannel(channel);
			retVal = false;
		}
 
	return retVal;
 
}


Indy already uses a 32K receive buffer by default, but you can always set
it higher (see the TIdIOHandler.RecvBufferSize property).
I made this 1,000,000 and it did not seem to change any thing. Is Indy supposed continue reading and Acking TCP data even if I fail to do a read? It does not seem to be doing so.


But how often are you actually calling it? And are you calling it in the
main thread, or in a worker thread? The fact that you have a client array
makes me think you are looping through the clients sequentially. Are you?
Or does each client run in its own thread?
I have a form which has 6 TIdTCPClients. The form reads them sequentially (the devices are synchronized) and passes the data to the rest of the system which does signal processing, graphing, data saving, etc.

Thanks,

Curtis
Curtis
Long Tran

Posts: 2
Registered: 6/22/15
Re: TIdTCPClient lags and times out  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 19, 2015 2:18 PM   in response to: Curtis Lending in response to: Curtis Lending
I have the same problem before too, but everything works out now.
Try this:
ChanTCPClient[chIndex]->IOHandler->ReadBytes(readBuffer, -1, true);
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdTCPClient lags and times out  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 20, 2015 11:03 PM   in response to: Long Tran in response to: Long Tran
Long wrote:

Try this:
ChanTCPClient[chIndex]->IOHandler->ReadBytes(readBuffer, -1, true);

Setting the AByteCount parameter to -1 tells ReadBytes() to return whatever
bytes are currently available at that moment. It waits up to the ReadTimeout
for any new bytes to arrive, and if they do then they are stored in the InputBuffer,
and then it returns everything that is currently in the InputBuffer. So,
if you have a short ReadTimeout, or data is arriving in quick intervals,
ReadBytes() would return arbitrary amounts of bytes fairly quickly.

--
Remy Lebeau (TeamB)
Curtis Lending

Posts: 33
Registered: 11/24/14
Re: TIdTCPClient lags and times out  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 30, 2015 2:36 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy,

I have been on vacation and finally got back to this.

Setting TIdIOHandler.RecvBufferSize to a huge size seems to have fixed my problem. I did not try a big enough number earlier.

Thanks,

Curtis
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02