Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: Sending multi-line message from server and read them on client?


This question is answered.


Permlink Replies: 8 - Last Post: Nov 15, 2016 10:23 AM Last Post By: Remy Lebeau (Te...
Ahmed Sayed

Posts: 173
Registered: 8/9/07
Sending multi-line message from server and read them on client?  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Nov 14, 2016 7:51 AM
Hello,

I am trying to send multiple messages that contains multi-line to
one of the connected clients, via "IOHandler->Write" on the server side
and on the client side i am reading incoming data in a TTimer, by using
InputAsString. but this returns the whole data read from the socked at
that time and sometimes the messages are currupted.

The message lines count are in the range from 4 to 8 lines, so there is no
specific number that i can define to split incoming data accordingly.

So, how can make the server send like 10 messages one after the other,
and then make the client read each message one by one.

I have tried ReadLn but it reads only one line.

What i mean is how to define a message start and end on server, and
make the client detects these starting and ending points.

Thanks in advance.

--
The limits of my language mean the limits of my world
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Sending multi-line message from server and read them on client? [Edit]
Correct
Click to report abuse...   Click to reply to this thread Reply
  Posted: Nov 14, 2016 10:44 AM   in response to: Ahmed Sayed in response to: Ahmed Sayed
Ahmed wrote:

I am trying to send multiple messages that contains multi-line to
one of the connected clients, via "IOHandler->Write" on the server
side and on the client side i am reading incoming data in a TTimer,
by using InputAsString.

That is not going to work. You need to delimit your messages and lines so
you know where one ends and the next begins. Why are you not using TIdIOHandler::WriteLn()
with TIdIOHandler::ReadLn(), or TIdIOHandler::Write(TStrings, True) with
TIdIOHandler::ReadStrings()? What do your messages actually look like?

but this returns the whole data read from the socked at
that time and sometimes the messages are currupted.

They are not corrupted. Your approach is simply not taking TCP streaming
and line/message boundaries into account, so the raw buffer you are reading
only contains portions of data.

The message lines count are in the range from 4 to 8 lines, so
there is no specific number that i can define to split incoming
data accordingly.

If you know ahead of time how many lines you are going to send, then send
the line count first before sending the actual lines. TIdIOHandler::Write(TStrings)
and TIdIOHandler::ReadStrings() handle line counts for you.

If you don't know ahead of time how many lines you are going to send, then
you have to send a unique delimiter after sending the last line, before starting
a new message. Then the reader would have to keep reading until it encounters
that delimiter.

So, how can make the server send like 10 messages one after the other,
and then make the client read each message one by one.

TCP is a byte stream. You HAVE to be explicit about your data boundaries!
That requires either 1) sending fixed-length data, 2) sending the data length
before sending the data, or 3) sending a unique terminator after the data.

I have tried ReadLn but it reads only one line.

Yes, but you can call it in a loop (or let ReadStrings() do that for you).
You should have one ReadLn() for each WriteLn().

What i mean is how to define a message start and end on server,
and make the client detects these starting and ending points.

Are you able to make changes to your protocol? Are you using a text-based
protocol or a binary protocol? A binary protocol is easier to mark boundaries
with (for instance, using STX and ETX bytes), but even text-based protocols
have ways of doing it, too (text-based protocols like POP3/SMTP, HTTP, etc
deal with variable-length data all the time).

--
Remy Lebeau (TeamB)
Ahmed Sayed

Posts: 173
Registered: 8/9/07
Re: Sending multi-line message from server and read them on client? [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Nov 14, 2016 11:07 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Thanks for the reply.

The data is json and i don't know exactly how many lines
it will be.

I am just guessing how many lines, but it might be more or less.
Also i am using text-based protocol.

I will try Write(Strings,true) and ReadStrings() and let you know.

--
The limits of my language mean the limits of my world
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Sending multi-line message from server and read them on client?[Edit]
Helpful
Click to report abuse...   Click to reply to this thread Reply
  Posted: Nov 14, 2016 11:44 AM   in response to: Ahmed Sayed in response to: Ahmed Sayed
Ahmed wrote:

The data is json and i don't know exactly how many lines it will be.

Just like XML, JSON is not line-based data. An entire JSON document can
exist in a single line, or multiple lines. Whitespace and line breaks are
only used for human readibility, but are not necessary for machine readibility.

If you know the total length of the JSON ahead of time, you can send that
length before sending the JSON itself. JSON is typically encoded in UTF-8,
so you could store the JSON in a UTF-8 encoded TStream and then use TIdIOHandler::Write(TStream),
which has an option of sending the TStream size for you (as either an Integer
or an Int64). And then on the receiving end, you can use ReadStream() or
ReadString(ReadInteger()), either of which will account for the preceding
length.

And just like XML, JSON is a self-contained, self-terminating format, so
even if you don't know the total length ahead of time (because you are streaming
the JSON as it is being generated), it is still possible for the receiver
to detect the end of the JSON document when it reaches the final closing
bracket. Though it would be best to actually send an end-of-data delimiter
after sending the JSON, to make sure.

Also i am using text-based protocol.

Then I strongly suggest you either:

1) send a line containing the textual representation of the JSON length,
before sending the JSON itself. The receiver can read the length line, convert
the text to an integer, and then read the specified number of bytes/characters.

2) send the JSON first, and then send a unique terminator that does not appear
in the JSON itself, such as a period ('.') on a line by itself. The receiver
can read data until it encounters that terminator. Either TIdIOHandler::ReadLn()
or TIdIOHandler::WaitFor() can be used for that purpose.

I will try Write(Strings,true) and ReadStrings() and let you know.

Please don't send JSON as line-based data.

--
Remy Lebeau (TeamB)
Ahmed Sayed

Posts: 173
Registered: 8/9/07
Re: Sending multi-line message from server and read them on client?[Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Nov 14, 2016 12:07 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Thanks I tried Write(Strings,true) and ReadStrings() and
it worked fine. but what do you mean by

"Please don't send JSON as line-based data."

The JSON Data will like this:
{
"UserName":"Ahmed Sayed",
"Age":31,
"Active": True
}


Is this wrong in anyway?

--
The limits of my language mean the limits of my world
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Sending multi-line message from server and read them onclient?[Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Nov 14, 2016 4:09 PM   in response to: Ahmed Sayed in response to: Ahmed Sayed
Ahmed wrote:

Thanks I tried Write(Strings,true) and ReadStrings() and it
worked fine. but what do you mean by

"Please don't send JSON as line-based data."

JSON is not based on lines, so using a TStrings to represent JSON data is
not the right thing to do.

--
Remy Lebeau (TeamB)
Ahmed Sayed

Posts: 173
Registered: 8/9/07
Re: Sending multi-line message from server and read them onclient?[Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Nov 15, 2016 4:39 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
So, what other techniques should i use that works best with
JSON.

--
The limits of my language mean the limits of my world
Sean Hoffman

Posts: 126
Registered: 3/28/99
Re: Sending multi-line message from server and read them onclient?[Edit]
Helpful
Click to report abuse...   Click to reply to this thread Reply
  Posted: Nov 15, 2016 5:27 AM   in response to: Ahmed Sayed in response to: Ahmed Sayed
Ahmed Sayed wrote:
So, what other techniques should i use that works best with
JSON.

--
The limits of my language mean the limits of my world

If you don't want to change the communication protocol, I would base64 encode the JSON and slap a CR NL on the end of the base64.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Sending multi-line message from server and read themonclient?[Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Nov 15, 2016 10:23 AM   in response to: Ahmed Sayed in response to: Ahmed Sayed
Ahmed wrote:

So, what other techniques should i use that works best with JSON.

I already mentioned the available options. Basically, you need to either
send the JSON size before sending the JSON itself, or else send a unique
terminator after sending the JSON.

--
Remy Lebeau (TeamB)
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02