Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: TIdMessage bug with In-Reply-To


This question is answered.


Permlink Replies: 4 - Last Post: Sep 12, 2016 6:46 PM Last Post By: Remy Lebeau (Te... Threads: [ Previous | Next ]
John May

Posts: 81
Registered: 6/25/10
TIdMessage bug with In-Reply-To  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Mar 3, 2016 1:44 PM
boost::scoped_ptr<TIdMessage> IdMsg(new TIdMessage(this));
TIdEMailAddressItem* ai;
 
IdMsg->From->Name	 = "John Smith";
IdMsg->From->Address = "john.smith@domain.com";
 
ai = IdMsg->Recipients->Add();
ai->Name	= "Mary Smith";
ai->Address = "mary.smith@domain.com";
 
IdMsg->Subject		= "Test";
IdMsg->MsgId		= "1";
IdMsg->InReplyTo	= "";
IdMsg->References	= "";
 
IdMsg->ExtraHeaders->Values["Message-Id"] = IdMsg->MsgId;
 
IdSMTP1->Host = "localhost";
IdSMTP1->Connect();
IdSMTP1->Send(IdMsg.get());
IdSMTP1->DisconnectNotifyPeer();


Output:

From: "John Smith" <john.smith@domain.com>
Subject: Test
To: "Mary Smith" <mary.smith@domain.com>
Date: Thu, 3 Mar 2016 22:38:54 +0100
Message-Id: <1>
In-Reply-To: <1>


Note the duplicated Message-Id in In-Reply-To. In-Reply-To should be non-existent as this is a new message.

Also a question - when using references to "reply-to" message is this enough?

// IdMsgOld = old message to which it is being replied to (already preloaded with content from old message)
// IdMsgReplyTo = new message which is a reply to old message
 
// references to reply-to message:
IdMsgReplyTo->MsgId	= "NewIDGeneratedHereUsingMyAlgorithm";
IdMsgReplyTo->InReplyTo	= IdMsgOld->MsgId; // Use old message ID as "In-Reply-To" header
IdMsgReplyTo->References	= IdMsgOld->References + " " + IdMsgReplyTo->MsgId; // Most recent ID is last, separated by space


The question is - does Indy automatically limit references to 3 entries (parent, grandparent, great-grandparent) or do I need to filter the IdMsgOld->References to exclude the starting entries until I have only 2 of them before adding the last one? And also, does Indy use space as separator between References entries? Or in short - how do I build InReplyTo and References list - is the above code sufficient?
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdMessage bug with In-Reply-To [Edit]
Helpful
Click to report abuse...   Click to reply to this thread Reply
  Posted: Mar 3, 2016 2:20 PM   in response to: John May in response to: John May
John wrote:

Note the duplicated Message-Id in In-Reply-To. In-Reply-To should be
non-existent as this is a new message.

That is intentional behavior. See the code in TIdMessage.GenerateHeader():

  {Generate In-Reply-To if at all possible to pacify SA.  Do this after FExtraHeaders
   added in case there is a message-ID present as an extra header.}
  if InReplyTo = '' then begin
    if FLastGeneratedHeaders.Values['Message-ID'] <> '' then begin  {do not 
localize}
      FLastGeneratedHeaders.Values['In-Reply-To'] := FLastGeneratedHeaders.Values['Message-ID']; 
 {do not localize}
    end else begin
     {CC: The following was originally present, but it so wrong that it has 
to go!}
     //Values['In-Reply-To'] := Subject;   {do not localize}
    end;
  end else begin
    FLastGeneratedHeaders.Values['In-Reply-To'] := InReplyTo; {do not localize}
  end;


Clients are not expected to generate their own message IDs, servers should
generate them when accepting the email, so the presence of a non-blank MsgId
assumes the email is a reply to an earlier email. Yes, I am aware that this
is not always the case.

// references to reply-to message:
IdMsgReplyTo->MsgId = "NewIDGeneratedHereUsingMyAlgorithm";
IdMsgReplyTo->InReplyTo = IdMsgOld->MsgId; // Use old message ID as "In-Reply-To"
header
IdMsgReplyTo->References = IdMsgOld->References + " " + IdMsgReplyTo->MsgId;
// Most recent ID is last, separated by space

Basically, yes. Though IdMsgReplyTo->MsgId should be IdMsgOld->MsgId (if
it is not already in the list):

The "References:" field will contain the contents of the parent's
"References:" field (if any) followed by the contents of the parent's
"Message-ID:" field (if any). If the parent message does not contain
a "References:" field but does have an "In-Reply-To:" field
containing a single message identifier, then the "References:" field
will contain the contents of the parent's "In-Reply-To:" field
followed by the contents of the parent's "Message-ID:" field (if
any). If the parent has none of the "References:", "In-Reply-To:",
or "Message-ID:" fields, then the new message will have no
"References:" field.

And you might want to check if IdMsgOld->References is not blank before inserting
a space character.

The question is - does Indy automatically limit references to 3 entries
(parent, grandparent, great-grandparent)

No.

do I need to filter the IdMsgOld->References to exclude the starting
entries until I have only 2 of them before adding the last one?

Yes, if you want to impose such limits. The relevant RFCs do not limit it
to 3 entries.

And also, does Indy use space as separator between References entries?

Neither. It is just a String value, its content is arbitrary to Indy, no
parsing is done on it at all.

--
Remy Lebeau (TeamB)
John May

Posts: 81
Registered: 6/25/10
Re: TIdMessage bug with In-Reply-To [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Mar 4, 2016 4:43 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:
That is intentional behavior. See the code in TIdMessage.GenerateHeader():
Clients are not expected to generate their own message IDs, servers should
generate them when accepting the email, so the presence of a non-blank MsgId
assumes the email is a reply to an earlier email. Yes, I am aware that this
is not always the case.

Can you please lift this restriction or make a workaround possibility? New message should have message id, generated by the client, without In-reply-to (being not present).

This restriction does no good and here are the arguments:

- all email programs generate message ID (with NO in-reply-to) for a new message - Microsoft Outlook, Mozilla Thunderbird, Outlook Express, Windows Mail, Windows Live Mail and for the reply they all utilize In-Reply-To and References.

- a good number of servers will reject your message if it is without message id (i had such a case too and I am surely not the only one) - http://stackoverflow.com/questions/4496821/sending-mail-but-no-message-id - http://forums2.atozed.com/viewtopic.php?f=7&t=6379 - so again, it should be generated by the client to be on the safe side. When however the message-id is generated, I have not seen servers rejecting the message. It is very easy to generate globally unique message-id in the client, server is not required for this and the algorithms are readily available on at least a few sites.

- no RFC actually specify that server is supposed to generate message ID. It can be done by client or by the server - https://en.wikipedia.org/wiki/Message-ID - "Message-IDs, if present, are generated by the client program sending the email (mail user agent, or MUA) or by the first mail server (mail submission agent, or MSA).[5]" - 3.6.4. Identification fields section of rfc2822 never mentions anything about message id must be generated by the server.

- rfc2822 says - The "In-Reply-To:" field may be used to identify the message (or messages) to which the new message is a reply. The "In-Reply-To:" field will contain the contents of the "Message- ID:" field of the message to which this one is a reply (the "parent message"). ---- it may be possibly harmful to put the exact same message id in the "In-Reply-To". New message should have only message-id and replies should have message-id + in-reply-to (which is NOT the same) + references. The presence of the in-reply-to suggest the message is a reply (which a new message is not). this is not desired behavior.

- it does not make sense at all that "in-reply-to" is the same if it was left blank. Why not leaving it alone as the user specified? You are never replying to the same message with the same ID - it creates circular reference which may only create issues with some mail programs.

Thank you for your other reply.
John May

Posts: 81
Registered: 6/25/10
Re: TIdMessage bug with In-Reply-To [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Mar 5, 2016 9:59 AM   in response to: John May in response to: John May
Additionally rfc2822 has example "conversation" between two senders which clearly shows how to handle MessageID (Appendix "A.2. Reply messages"):
http://www.ietf.org/rfc/rfc2822.txt

The first message has ONLY message ID, no In-Reply-To which is blank.

Clearly from that example, allowing Indy to generate In-reply-to which is the same like message-id is a bug.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdMessage bug with In-Reply-To
Correct
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 12, 2016 6:46 PM   in response to: John May in response to: John May
John May wrote:
Note the duplicated Message-Id in In-Reply-To. In-Reply-To should be non-existent as this is a new message.

As a followup for this - TIdMessage has now been updated with logic changes in how it handles the "Message-ID" and "In-Reply-To" headers:

http://indyproject.org/sockets/blogs/changelog/20160912.aspx

In a nutshell, the TIdMessage.MsgId property now generates a "Message-ID" header when creating a new email, and the "In-Reply-To" header is no longer auto-generated when the TIdMessage.InReplyTo property is blank.

--
Remy Lebeau (TeamB)
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02