Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: TIdAttachmentMemory...MTOM/XOP



Permlink Replies: 32 - Last Post: Jul 8, 2015 10:26 AM Last Post By: Quentin Correll
David Keith

Posts: 196
Registered: 12/10/99
TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Mar 17, 2015 10:25 AM
I've been tasked with researching and prototyping a method to use
MTOM/XOP
(http://en.wikipedia.org/wiki/Message_Transmission_Optimization_Mechanism,
http://en.wikipedia.org/wiki/XML-binary_Optimized_Packaging) to wrap
both XML docs and binary MIME attachments for sending to exchange partners.

I've been reviewing Indy's MIME handling capabilities, as I have used
them on several occasions for building email messages with MIME encoded
attachments, for potential use with MTOM/XOP packaging.

My concern is that by default Indy base64 encodes the binary data in
order for the message to be processable by SMTP servers. This is exactly
the reason why MTOM/XOP was developed, to reduce processing overhead and
XML packet size with very large docs with attachments.

TIdAttachmentMemory, I noticed, doesn't base64 encode the binary data,
which tells me that it must occur elsewhere. I'm wondering if there is a
way that I could 'hybrid' Indy's MIME processing capabilities for this task.

Here is an example of what an XML message with MTOM/XOP packaging looks
like (sans binary data, of course):
http://wiki.ihe.net/index.php?title=XDS.b_Implementation#Example_Provide_and_Register_Document_Set-b_transaction_.28with_full_metadata.29

Would Indy be a viable option for something like this? Or would I need
to code something completely custom for this?

TIA
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Mar 17, 2015 11:04 AM   in response to: David Keith in response to: David Keith
David Keith wrote:

I've been tasked with researching and prototyping a method to use
MTOM/XOP
(http://en.wikipedia.org/wiki/Message_Transmission_Optimization_Mechanism,
http://en.wikipedia.org/wiki/XML-binary_Optimized_Packaging) to wrap
both XML docs and binary MIME attachments for sending to exchange partners.

I've been reviewing Indy's MIME handling capabilities, as I have used
them on several occasions for building email messages with MIME encoded
attachments, for potential use with MTOM/XOP packaging.

My concern is that by default Indy base64 encodes the binary data in
order for the message to be processable by SMTP servers. This is exactly
the reason why MTOM/XOP was developed, to reduce processing overhead and
XML packet size with very large docs with attachments.

TIdAttachmentMemory, I noticed, doesn't base64 encode the binary data,
which tells me that it must occur elsewhere. I'm wondering if there is a
way that I could 'hybrid' Indy's MIME processing capabilities for this task.

Here is an example of what an XML message with MTOM/XOP packaging looks
like (sans binary data, of course):
http://wiki.ihe.net/index.php?title=XDS.b_Implementation#Example_Provide_and_Register_Document_Set
-b_transaction_.28with_full_metadata.29

Would Indy be a viable option for something like this? Or would I need
to code something completely custom for this?

TIA

Just to clarify, the goal is to add MIME attachment sections without base64 encoding attachments -
just add a MIME section for the attachments, and add the binary content without first base64
encoding the binary content.

--
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Mar 19, 2015 11:54 AM   in response to: David Keith in response to: David Keith
David wrote:

TIdAttachmentMemory, I noticed, doesn't base64 encode the binary data,
which tells me that it must occur elsewhere.

Correct. It is handled by TIdMessageClient, based on the attachment's ContentTransfer
property, which can be set to 'base64' (default), 'quoted-printable', 'binhex40',
'8bit', or 'binary'.

Here is an example of what an XML message with MTOM/XOP packaging
looks like (sans binary data, of course):
<snip>

If you are trying to send an MTOM/XOP encoded POST request to an HTTP server
using Indy, I would not suggest using TIdMessage/TIdMessageParts for that.
They encode data according to SMTP rules, which is not suitable for HTTP.
Also, the TIdHTTP.Post() overload that accepts a TIdMultipartFormDataStream
is not suitable for MTOM/XOP either, as it is intended for HTML webform posts.
So you are best off simply formatting the MIME data manually, storing it
in a TStream of your choosing, and then pass that to TIdHTTP.Post() for it
to send as-is. For example:

var
  Boundary: string;
  ContentID: string;
  SoapAction: string;
  MStrm: TMemoryStream;
begin
  Boundary := 'MIMEBoundaryurn_uuid_566EAD10FEBB55C5A61257193478449';
  ContentID := '<0.urn:uuid:566EAD10FEBB55C5A61257193478450@apache.org>';
  SoapAction := 'urn:ihe:iti:2007:ProvideAndRegisterDocumentSet-b';
 
  MStrm := TMemoryStream.Create;
  try
    WriteStringToStream(MStrm, '--' + Boundary + EOL);
    WriteStringToStream(MStrm, 'Content-Type: application/xop+xml; charset=UTF-8; 
type="application/soap+xml"' + EOL);
    WriteStringToStream(MStrm, 'Content-Transfer-Encoding: binary' + EOL);
    WriteStringToStream(MStrm, 'Content-ID: ' + ContentID + EOL);
    WriteStringToStream(MStrm, EOL);
    // .. write UTF-8 encoded XML to MStrm as needed...
    WriteStringToStream(MStrm, EOL + '--' + Boundary + EOL);
    WriteStringToStream(MStrm, 'Content-Type: text/plain' + EOL);
    WriteStringToStream(MStrm, 'Content-Transfer-Encoding: binary' + EOL);
    WriteStringToStream(MStrm, 'Content-ID: ' + ... + EOL);
    WriteStringToStream(MStrm, EOL);
    // .. write document to MStrm as needed...
    WriteStringToStream(MStrm, EOL + '--' + Boundary + '--' + EOL);
    MStrm.Position := 0;
 
    IdHTTP1.Request.ContentType := 'multipart/related; boundary=' + Boundary 
+ '; type="application/xop+xml"; start="' + ContentID + '"; start-info="application/soap+xml"; 
action=" + SoapAction + "';
    IdHTTP1.Request.UserAgent := 'Axis2';
    IdHTTP1.Post('http://localhost:5000/tf6/services/xdsrepositoryb', MStrm);
  finally
    MStrm.Free;
  end;
end;


If you want something a little more generic, you can incorprate the TIdMessageParts
collection into this code so you can identify documents and attachments more
dynamically, but you will have to loop through the collection encoding the
MIME portions manually, similar to how TIdMessageClient does (without the
SMTP and ContentTransfer portions, of course). Indy's MIME framework is
currently very tied to email usage only, it is not meant for general purpose
use. That will change in Indy 11, but that is a ways off.

--
Remy Lebeau (TeamB)
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Mar 20, 2015 2:30 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:

David wrote:

TIdAttachmentMemory, I noticed, doesn't base64 encode the binary data,
which tells me that it must occur elsewhere.

Correct. It is handled by TIdMessageClient, based on the attachment's ContentTransfer
property, which can be set to 'base64' (default), 'quoted-printable', 'binhex40',
'8bit', or 'binary'.

Here is an example of what an XML message with MTOM/XOP packaging
looks like (sans binary data, of course):
<snip>

If you are trying to send an MTOM/XOP encoded POST request to an HTTP server
using Indy, I would not suggest using TIdMessage/TIdMessageParts for that.
They encode data according to SMTP rules, which is not suitable for HTTP.
Also, the TIdHTTP.Post() overload that accepts a TIdMultipartFormDataStream
is not suitable for MTOM/XOP either, as it is intended for HTML webform posts.
So you are best off simply formatting the MIME data manually, storing it
in a TStream of your choosing, and then pass that to TIdHTTP.Post() for it
to send as-is. For example:

var
  Boundary: string;
  ContentID: string;
  SoapAction: string;
  MStrm: TMemoryStream;
begin
  Boundary := 'MIMEBoundaryurn_uuid_566EAD10FEBB55C5A61257193478449';
  ContentID := '<0.urn:uuid:566EAD10FEBB55C5A61257193478450@apache.org>';
  SoapAction := 'urn:ihe:iti:2007:ProvideAndRegisterDocumentSet-b';
 
  MStrm := TMemoryStream.Create;
  try
    WriteStringToStream(MStrm, '--' + Boundary + EOL);
    WriteStringToStream(MStrm, 'Content-Type: application/xop+xml; charset=UTF-8; 
type="application/soap+xml"' + EOL);
    WriteStringToStream(MStrm, 'Content-Transfer-Encoding: binary' + EOL);
    WriteStringToStream(MStrm, 'Content-ID: ' + ContentID + EOL);
    WriteStringToStream(MStrm, EOL);
    // .. write UTF-8 encoded XML to MStrm as needed...
    WriteStringToStream(MStrm, EOL + '--' + Boundary + EOL);
    WriteStringToStream(MStrm, 'Content-Type: text/plain' + EOL);
    WriteStringToStream(MStrm, 'Content-Transfer-Encoding: binary' + EOL);
    WriteStringToStream(MStrm, 'Content-ID: ' + ... + EOL);
    WriteStringToStream(MStrm, EOL);
    // .. write document to MStrm as needed...
    WriteStringToStream(MStrm, EOL + '--' + Boundary + '--' + EOL);
    MStrm.Position := 0;
 
    IdHTTP1.Request.ContentType := 'multipart/related; boundary=' + Boundary 
+ '; type="application/xop+xml"; start="' + ContentID + '"; start-info="application/soap+xml"; 
action=" + SoapAction + "';
    IdHTTP1.Request.UserAgent := 'Axis2';
    IdHTTP1.Post('http://localhost:5000/tf6/services/xdsrepositoryb', MStrm);
  finally
    MStrm.Free;
  end;
end;


If you want something a little more generic, you can incorprate the TIdMessageParts
collection into this code so you can identify documents and attachments more
dynamically, but you will have to loop through the collection encoding the
MIME portions manually, similar to how TIdMessageClient does (without the
SMTP and ContentTransfer portions, of course). Indy's MIME framework is
currently very tied to email usage only, it is not meant for general purpose
use. That will change in Indy 11, but that is a ways off.

Thanks for the response, Remy. Informative as always.

I had kind of settled on doing it this way, just been trying to find SOME kind of D tech that would
simplify this process a bit, maybe make it a bit more elegant. This will have to do.

<grumble grumble>It's very unfortunate that D has 'supported' SOAP v1.2 since what... XE? Yet
MTOM/XOP, which is a part of the standard, doesn't appear to be supported. Yet .NET & java happily
support this stuff without issue...</grumble grumble>

Thanks again, Remy.

--

David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: May 4, 2015 10:32 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:

David wrote:

TIdAttachmentMemory, I noticed, doesn't base64 encode the binary data,
which tells me that it must occur elsewhere.

Correct. It is handled by TIdMessageClient, based on the attachment's ContentTransfer
property, which can be set to 'base64' (default), 'quoted-printable', 'binhex40',
'8bit', or 'binary'.

Here is an example of what an XML message with MTOM/XOP packaging
looks like (sans binary data, of course):
<snip>

If you are trying to send an MTOM/XOP encoded POST request to an HTTP server
using Indy, I would not suggest using TIdMessage/TIdMessageParts for that.
They encode data according to SMTP rules, which is not suitable for HTTP.
Also, the TIdHTTP.Post() overload that accepts a TIdMultipartFormDataStream
is not suitable for MTOM/XOP either, as it is intended for HTML webform posts.
So you are best off simply formatting the MIME data manually, storing it
in a TStream of your choosing, and then pass that to TIdHTTP.Post() for it
to send as-is. For example:

var
  Boundary: string;
  ContentID: string;
  SoapAction: string;
  MStrm: TMemoryStream;
begin
  Boundary := 'MIMEBoundaryurn_uuid_566EAD10FEBB55C5A61257193478449';
  ContentID := '<0.urn:uuid:566EAD10FEBB55C5A61257193478450@apache.org>';
  SoapAction := 'urn:ihe:iti:2007:ProvideAndRegisterDocumentSet-b';
 
  MStrm := TMemoryStream.Create;
  try
    WriteStringToStream(MStrm, '--' + Boundary + EOL);
    WriteStringToStream(MStrm, 'Content-Type: application/xop+xml; charset=UTF-8; 
type="application/soap+xml"' + EOL);
    WriteStringToStream(MStrm, 'Content-Transfer-Encoding: binary' + EOL);
    WriteStringToStream(MStrm, 'Content-ID: ' + ContentID + EOL);
    WriteStringToStream(MStrm, EOL);
    // .. write UTF-8 encoded XML to MStrm as needed...
    WriteStringToStream(MStrm, EOL + '--' + Boundary + EOL);
    WriteStringToStream(MStrm, 'Content-Type: text/plain' + EOL);
    WriteStringToStream(MStrm, 'Content-Transfer-Encoding: binary' + EOL);
    WriteStringToStream(MStrm, 'Content-ID: ' + ... + EOL);
    WriteStringToStream(MStrm, EOL);
    // .. write document to MStrm as needed...
    WriteStringToStream(MStrm, EOL + '--' + Boundary + '--' + EOL);
    MStrm.Position := 0;
 
    IdHTTP1.Request.ContentType := 'multipart/related; boundary=' + Boundary 
+ '; type="application/xop+xml"; start="' + ContentID + '"; start-info="application/soap+xml"; 
action=" + SoapAction + "';
    IdHTTP1.Request.UserAgent := 'Axis2';
    IdHTTP1.Post('http://localhost:5000/tf6/services/xdsrepositoryb', MStrm);
  finally
    MStrm.Free;
  end;
end;


If you want something a little more generic, you can incorprate the TIdMessageParts
collection into this code so you can identify documents and attachments more
dynamically, but you will have to loop through the collection encoding the
MIME portions manually, similar to how TIdMessageClient does (without the
SMTP and ContentTransfer portions, of course). Indy's MIME framework is
currently very tied to email usage only, it is not meant for general purpose
use. That will change in Indy 11, but that is a ways off.

In the code above you write everything into the stream which is all good. If I write this stream
out to an AnsiString, it's all text, no binary. So I guess my question is, does the determination
of whether or not any part of the content gets encoded as binary for the transfer, get determined
by the attribute 'Content-Transfer-Encoding: binary'?

In other words even though everything written to the stream gets translated into it's numeric
Ord(char) equivalent, for example the number '1' gets translated to #49 in the stream byte, but
isn't officially 'binary' until the IdHTTP component gets ahold of it, sees the attribute
'Content-Transfer-Encoding: binary', and transfers that section of the stream in a binary format?
So everything that's not masked as binary encoding gets transferred as... text/ascii chars?

Just not quite sure how this works under the hood.

Thanks again.

--
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: May 4, 2015 11:00 AM   in response to: David Keith in response to: David Keith
David wrote:

In the code above you write everything into the stream which is all
good. If I write this stream out to an AnsiString, it's all text, no
binary.

No, it does contain binary as well, if you wrote binary data into the TStream
to begin with.

So I guess my question is, does the determination of whether or not
any part of the content gets encoded as binary for the transfer, get
determined by the attribute 'Content-Transfer-Encoding: binary'?

That header tells the receiver whether the body data for that given MIME
part is encoded as binary or not. But you still have to actually encode
the data accordingly when writing it to the TStream. For instance, if you
specify "Content-Transfer-Encoding: base64" then you can use the TIdEncoderMIME
class to encode the data as base64 when writing it to the TStream, eg:

MStrm := TMemoryStream.Create;
try
  WriteStringToStream(MStrm, '--' + Boundary + EOL);
  WriteStringToStream(MStrm, 'Content-Type: application/xop+xml; charset=UTF-8; 
type="application/soap+xml"' + EOL);
  WriteStringToStream(MStrm, 'Content-Transfer-Encoding: base64' + EOL);
  WriteStringToStream(MStrm, 'Content-ID: ' + ContentID + EOL);
  WriteStringToStream(MStrm, EOL);
  IdEncoderMIME.EncodeString(XML, MStrm, IndyTextEncoding_UTF8);
  WriteStringToStream(MStrm, EOL + '--' + Boundary + EOL);
  WriteStringToStream(MStrm, 'Content-Type: text/plain' + EOL);
  WriteStringToStream(MStrm, 'Content-Transfer-Encoding: base64' + EOL);
  WriteStringToStream(MStrm, 'Content-ID: ' + ... + EOL);
  WriteStringToStream(MStrm, EOL);
  IdEncoderMIME.EncodeString(Document, MStrm, IndyTextEncoding_ASCII);
  WriteStringToStream(MStrm, EOL + '--' + Boundary + '--' + EOL);
  MStrm.Position := 0;
  //...
finally
  MStrm.Free;
end;


In other words even though everything written to the stream gets
translated into it's numeric Ord(char) equivalent, for example the
number '1' gets translated to #49 in the stream byte, but isn't
officially 'binary' until the IdHTTP component gets ahold of it, sees
the attribute 'Content-Transfer-Encoding: binary', and transfers that
section of the stream in a binary format?

No, that is not how TIdHTTP works. When you Post() a TStream, the bytes
in that TStream get transmitted as-is to the HTTP server. It is the server,
not TIdHTTP, that then parses the MIME headers and interprets the body data
of each MIME part accordingly.

So everything that's not masked as binary encoding gets transferred
as... text/ascii chars?

No.

--
Remy Lebeau (TeamB)
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: May 4, 2015 12:38 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:

IndyTextEncoding_UTF8

Thanks for your reply Remy. Forgive me for my ignorance, but the whole reason that I'm trying to
figure this out is because I have to take a HUGE chunk of ascii (ansi) text, convert it to binary,
and post it to somebody else's server, with part of the content posted being as human readable XML
(soap envelope/header/body) and part of it being in unencoded binary with MIME boundaries. This to
save processing overhead on the receiver's side (whatever).

That's the advantage(?) to using MTOM/XOP encoded attachments - and the entire point - over base64
encoded binary content - less processing overhead on the server. So instead of base64 encoding the
binary content, we're leaving it in binary(?) and wrapping it inside of a MIME boundary, specifying
the conent-type as binary and providing the content length of the binary content.

Since this is implemented fully in .NET a C#/VB.NET developer doesn't have to sweat this or even
care.

Since this is not implemented in any way, shape or form in Delphi we have to figure out how to take
text and represent it as a binary attachment.

Years ago we would do this (and in some rare cases still today) with advanced component sets like
ASTA, RemObjects etc. using extremely efficient binary protocols with data that could be of any
type, fully typed or not typed at all, even using compression etc.

M$'s answer was of course to implement the entire world in XML. So take the least efficient
operating system the world has ever known, turn it into a significantly less efficient platform by
handling all structured data that is being transferred over the wire as XML. Which as most Delphi
developers the world over by default knew would create an even greater performance nightmare.

This nightmare in turn should have been captialized on by BorCoDero to market the awesome power of
Delphi N-Tier solutions and springboard Delphi center stage into the enterprise software
development world.

Instead our superior tech was subordinated to 'collaborative' efforts with the M$ beast for the
sake of... survival??

So here I am trying to figure out an extremely stupid way of doing something that I was able to do
18 years ago with ease, with a great deal of manual effort.

So please forgive me if I seem obtuse, but because of my past experience I'm having a bit of
difficulty wrapping my head around the 'binary encoding' part of this.

If writing text to a stream and transferring it to a server via http protocol within a SOAP
wrapper doesn't qualify as converting the text payload to binary, what approach does, exactly?? Do
you understand this? Please keep in mind the afore referenced MTOM/XOP Wiki.

I have code that converts string to binary (1's and 0's) but I'm not sure if that's what this means
(I doubt it) or is it something else?

Thanks again.

--
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: May 4, 2015 1:29 PM   in response to: David Keith in response to: David Keith
David wrote:

Forgive me for my ignorance, but the whole reason that I'm trying to figure
this out is because I have to take a HUGE chunk of ascii (ansi) text, convert
it to binary, and post it to somebody else's server, with part of the content
posted being as human readable XML (soap envelope/header/body) and part
of it being in unencoded binary with MIME boundaries.

The code I gave you does exactly that, except that it stores everything into
a single TStream before then posting it. TIdHTTP does not support live streaming,
so you cannot encode and post individual pieces, unless you implement a custom
TStream class, like Indy does for TIdMultipartFormDataStream. Otherwise,
you would have to switch to TIdTCPClient directly and implement HTTP manually,
then you can send the request data however you want.

If writing text to a stream and transferring it to a server via http
protocol within a SOAP wrapper doesn't qualify as converting the text
payload to binary, what approach does, exactly?? Do you understand
this? Please keep in mind the afore referenced MTOM/XOP Wiki.

"binary" simple refers to sending raw 8bit byte octets of the data without
any further encoding. So even text can be "binary" as its raw byte octets
would be sent as-is.

I have code that converts string to binary (1's and 0's)

That is not what MIME's "binary" encoding is about.

--
Remy Lebeau (TeamB)
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: May 11, 2015 8:20 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:

David wrote:

In the code above you write everything into the stream which is all
good. If I write this stream out to an AnsiString, it's all text, no
binary.

No, it does contain binary as well, if you wrote binary data into the TStream
to begin with.

So I guess my question is, does the determination of whether or not
any part of the content gets encoded as binary for the transfer, get
determined by the attribute 'Content-Transfer-Encoding: binary'?

That header tells the receiver whether the body data for that given MIME
part is encoded as binary or not. But you still have to actually encode
the data accordingly when writing it to the TStream. For instance, if you
specify "Content-Transfer-Encoding: base64" then you can use the TIdEncoderMIME
class to encode the data as base64 when writing it to the TStream, eg:

MStrm := TMemoryStream.Create;
try
  WriteStringToStream(MStrm, '--' + Boundary + EOL);
  WriteStringToStream(MStrm, 'Content-Type: application/xop+xml; charset=UTF-8; 
type="application/soap+xml"' + EOL);
  WriteStringToStream(MStrm, 'Content-Transfer-Encoding: base64' + EOL);
  WriteStringToStream(MStrm, 'Content-ID: ' + ContentID + EOL);
  WriteStringToStream(MStrm, EOL);
  IdEncoderMIME.EncodeString(XML, MStrm, IndyTextEncoding_UTF8);
  WriteStringToStream(MStrm, EOL + '--' + Boundary + EOL);
  WriteStringToStream(MStrm, 'Content-Type: text/plain' + EOL);
  WriteStringToStream(MStrm, 'Content-Transfer-Encoding: base64' + EOL);
  WriteStringToStream(MStrm, 'Content-ID: ' + ... + EOL);
  WriteStringToStream(MStrm, EOL);
  IdEncoderMIME.EncodeString(Document, MStrm, IndyTextEncoding_ASCII);
  WriteStringToStream(MStrm, EOL + '--' + Boundary + '--' + EOL);
  MStrm.Position := 0;
  //...
finally
  MStrm.Free;
end;


In other words even though everything written to the stream gets
translated into it's numeric Ord(char) equivalent, for example the
number '1' gets translated to #49 in the stream byte, but isn't
officially 'binary' until the IdHTTP component gets ahold of it, sees
the attribute 'Content-Transfer-Encoding: binary', and transfers that
section of the stream in a binary format?

No, that is not how TIdHTTP works. When you Post() a TStream, the bytes
in that TStream get transmitted as-is to the HTTP server. It is the server,
not TIdHTTP, that then parses the MIME headers and interprets the body data
of each MIME part accordingly.

So everything that's not masked as binary encoding gets transferred
as... text/ascii chars?

No.

Can TIdHTTP (Indy 10) handle chunked transfer encoding?

Thanks.

--
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: May 11, 2015 12:53 PM   in response to: David Keith in response to: David Keith
David wrote:

Can TIdHTTP (Indy 10) handle chunked transfer encoding?

When receiving responses, yes. Not when sending requests.

--
Remy Lebeau (TeamB)
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: May 11, 2015 3:45 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:

David wrote:

Can TIdHTTP (Indy 10) handle chunked transfer encoding?

When receiving responses, yes. Not when sending requests.

Thanks for letting me know (grumble grumble).

When setting the content length specifier for the post transaction, do I count the entire length of
the stream? Do I need to do that for the MIME content section separately?

Thanks again.

--
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: May 11, 2015 4:08 PM   in response to: David Keith in response to: David Keith
David wrote:

When setting the content length specifier for the post transaction, do
I count the entire length of the stream?

Yes, however TIdHTTP will handle that for you. You do not need to set the
ContentLength manually (TIdHTTP will just overwrite you).

Do I need to do that for the MIME content section separately?

Each MIME part may have its own Content-Length, but that is not commonly
used since MIME is self-terminating using its boundaries between parts.

--
Remy Lebeau (TeamB)
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 30, 2015 1:16 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:

David wrote:

In the code above you write everything into the stream which is all
good. If I write this stream out to an AnsiString, it's all text, no
binary.

No, it does contain binary as well, if you wrote binary data into the TStream
to begin with.

So I guess my question is, does the determination of whether or not
any part of the content gets encoded as binary for the transfer, get
determined by the attribute 'Content-Transfer-Encoding: binary'?

That header tells the receiver whether the body data for that given MIME
part is encoded as binary or not. But you still have to actually encode
the data accordingly when writing it to the TStream. For instance, if you
specify "Content-Transfer-Encoding: base64" then you can use the TIdEncoderMIME
class to encode the data as base64 when writing it to the TStream, eg:

Will this method work with a web service? Can I simply post using the correct URL and the web
service will accept it? Or do I need to use THTTPRIO (or the TIdHTTPRIO) code that is available
online?

The test server doesn't have a WSDL... http://transport-testing.nist.gov/ttt/

Thanks.
--

Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 30, 2015 1:38 PM   in response to: David Keith in response to: David Keith
David wrote:

Will this method work with a web service? Can I simply post using
the correct URL and the web service will accept it?

That depends on the web service, and its level of MIME compliance.

--
Remy Lebeau (TeamB)
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 30, 2015 2:40 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:
David wrote:

Will this method work with a web service? Can I simply post using
the correct URL and the web service will accept it?

That depends on the web service, and its level of MIME compliance.

The web service in question has a validator that allows you to upload a file and test it's contents. I've been building files with the full MIME MTOM/XOP setup and passing validation 100%. So the web service should be fully compliant...

I'm thinking that maybe because I'm just POSTing the data without wrapping it in web service client method calling language it's not able to receive it.

It returns this error:

<S:Envelope xmlns:S="http://www.w3.org/2003/05/soap-envelope">
   <S:Header/>
   <S:Body>
      <fault:Fault xmlns:fault="http://www.w3.org/2003/05/soap-envelope">
         <fault:Code>
            <fault:Value>fault:Sender</fault:Value>
         </fault:Code>
         <fault:Reason>
            <fault:Text xml:lang="en">Simulator: Do not understand the transaction requested by this endpoint (rb:12080) in http://transport-testing.nist.gov:12080/ttt/sim/62a05d13-a513-41d0-92d6-843c70b1c89b/reg/rb:12080/ttt/sim/90a1b516-4f03-4796-a342-84118f4e62ac/rec/xdrpr - Endpoint format is http://transport-testing.nist.gov:12080/ttt/sim/simid/actor/transaction[/validation] where simid, actor and transaction are variables for simulators. If validation is included, then this validation must be performed successfully for the transaction to be successful.  Validations are documented as part of tests that use them.[]</fault:Text>
         </fault:Reason>
      </fault:Fault>
   </S:Body>
</S:Envelope>
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 30, 2015 3:09 PM   in response to: David Keith in response to: David Keith
David wrote:

I'm thinking that maybe because I'm just POSTing the data without
wrapping it in web service client method calling language it's not
able to receive it.

I cannot answer that, as I cannot see the complete HTTP message you are actually
posting, or what the webservice is expecting. At the very least, use a Packet
sniffer like Wireshark, or attach a TIdLog... component to TIdHTTP, to capture
everything that TIdHTTP is actually sending over its socket. That will help
diagnose whether the HTTP messaging is conforming to the webservice's specifications
or not.

--
Remy Lebeau (TeamB)
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 30, 2015 4:08 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:
David wrote:

I'm thinking that maybe because I'm just POSTing the data without
wrapping it in web service client method calling language it's not
able to receive it.

I cannot answer that, as I cannot see the complete HTTP message you are actually
posting, or what the webservice is expecting. At the very least, use a Packet
sniffer like Wireshark, or attach a TIdLog... component to TIdHTTP, to capture
everything that TIdHTTP is actually sending over its socket. That will help
diagnose whether the HTTP messaging is conforming to the webservice's specifications
or not.

--
Remy Lebeau (TeamB)

Here's a snippet of the first part of the transmission:

POST /ttt/sim/62a05d13-a513-41d0-92d6-843c70b1c89b/reg/rb:12080/ttt/sim/90a1b516-4f03-4796-a342-84118f4e62ac/rec/xdrpr HTTP/1.0<EOL>
Connection: keep-alive<EOL>
Content-Type: multipart/related; boundary=CD8749D6047941CC846822C48C0D6C33; type="application/xop+xml"; start="<F2D1ECBB4DD04EF5886F44B86FD312EF>"; start-info="application/soap+xml"; action="urn:ihe:iti:2007:ProvideAndRegisterDocumentSet-b"<EOL>
Content-Length: 23758<EOL>
Host: transport-testing.nist.gov:12080<EOL>
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8<EOL>
Accept-Encoding: identity<EOL>User-Agent: Wellsoft ICMS<EOL><EOL>

Could the '<EOL>'s be a problem?
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jun 30, 2015 7:08 PM   in response to: David Keith in response to: David Keith
David wrote:

Here's a snippet of the first part of the transmission:

That is not really helpful without seeing the actual message body that goes
with it.

Could the '<EOL>'s be a problem?

No, those are simply how Indy's TIdLog... components log CRLFs when the ReplaceCRLF
property is true (which it is by default).

--
Remy Lebeau (TeamB)
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 1, 2015 8:12 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
--
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 1, 2015 8:50 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
--
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 1, 2015 8:52 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
--
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 1, 2015 8:59 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:
David wrote:

Here's a snippet of the first part of the transmission:

That is not really helpful without seeing the actual message body that goes
with it.

Could the '<EOL>'s be a problem?

No, those are simply how Indy's TIdLog... components log CRLFs when the ReplaceCRLF
property is true (which it is by default).

--
Remy Lebeau (TeamB)
Remy Lebeau (TeamB) wrote:

David wrote:

Here's a snippet of the first part of the transmission:

That is not really helpful without seeing the actual message body that goes
with it.

Could the '<EOL>'s be a problem?

No, those are simply how Indy's TIdLog... components log CRLFs when the ReplaceCRLF
property is true (which it is by default).

In building the content for transmission I did the entire MTOM/XOP thing and attached two documents
inside of MIME parts. When I pulled out the content from TIdLogStream, the MIME attachments aren't
there. The MIME sections are there, but they do not contain the documents. Here is the text:

POST
/ttt/sim/62a05d13-a513-41d0-92d6-843c70b1c89b/reg/rb:12080/ttt/sim/90a1b516-4f03-4796-a342-84118f4e6
2ac/rec/xdrpr HTTP/1.0
Connection: keep-alive
Content-Type: multipart/related; boundary=A55D8908BAE949D88C1C22D5B47C6C04;
type="application/xop+xml"; start="<ECA99A17D89C4F6F8D73A2C5525CBECD>";
start-info="application/soap+xml"; action="urn:ihe:iti:2007:ProvideAndRegisterDocumentSet-b"
Content-Length: 23758
Host: transport-testing.nist.gov:12080
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: identity
User-Agent: Wellsoft ICMS

--A55D8908BAE949D88C1C22D5B47C6C04
Content-Type: <application/xop+xml; charset=UTF-8; type="application/soap+xml">
Content-Transfer-Encoding: binary
Content-ID: <ECA99A17D89C4F6F8D73A2C5525CBECD>

<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope"
xmlns:wsa="http://www.w3.org/2005/08/addressing">
  <soapenv:Header>
    <wsa:To soapenv:mustUnderstand="true"/>
    <wsa:MessageID>06A7857186054500AB6528783863A4B6</wsa:MessageID>
    <wsa:Action
soapenv:mustUnderstand="true">urn:ihe:iti:2007:ProvideAndRegisterDocumentSet-b</wsa:Action>
  </soapenv:Header>
  <soapenv:Body>
    <!--********************Start of Metadata********************-->
    <xdsb:ProvideAndRegisterDocumentSetRequest xmlns:xdsb="urn:ihe:iti:xds-b:2007">
      <lcm:SubmitObjectsRequest xmlns:lcm="urn:oasis:names:tc:ebxml-regrep:xsd:lcm:3.0"
xmlns:rim="urn:oasis:names:tc:ebxml-regrep:xsd:rim:3.0">
        <rim:RegistryObjectList>
          <rim:RegistryPackage objectType="urn:uuid:a54d6aa5-d40d-43f9-88c5-b4633d873bdd"
id="urn:uuid:d2d7eb19-c5f0-496d-ba47-2be5801a8b41">
            <rim:Slot name="submissionTime">
              <rim:ValueList>
                <rim:Value>20150701144357</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Name>
              <rim:LocalizedString value="Physical"/>
            </rim:Name>
            <rim:Description>
              <rim:LocalizedString value="Annual Physical"/>
            </rim:Description>
            <rim:Classification
classificationScheme="urn:uuid:a7058bb9-b4e4-4307-ba5b-e3f0ab85e12d" id="2.16.840.1.113883.3.809"
classifiedObject="urn:uuid:d2d7eb19-c5f0-496d-ba47-2be5801a8b41">
              <rim:Slot name="authorSpecialty">
                <rim:ValueList>
                  <rim:Value>Emergency Medicine</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Slot name="authorRole">
                <rim:ValueList>
                  <rim:Value>ED Attending</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Slot name="authorInstitution">
                <rim:ValueList>
                  <rim:Value>Robert Wood Johnson University Hospital</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Slot name="authorPerson">
                <rim:ValueList>
                  <rim:Value>^Robert Wood Johnson^University Hospital</rim:Value>
                </rim:ValueList>
              </rim:Slot>
            </rim:Classification>
            <rim:Classification
classificationScheme="urn:uuid:aa543740-bdda-424e-8c96-df4873be8500" nodeRepresentation="History
and Physical" id="cl09" classifiedObject="urn:uuid:d2d7eb19-c5f0-496d-ba47-2be5801a8b41">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>Connect-a-thon contentTypeCodes</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="History and Physical"/>
              </rim:Name>
            </rim:Classification>
            <rim:ExternalIdentifier registryObject="urn:uuid:d2d7eb19-c5f0-496d-ba47-2be5801a8b41"
identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8" id="ei03"
value="129.6.58.42.33726">
              <rim:Name>
                <rim:LocalizedString value="XDSSubmissionSet.uniqueId"/>
              </rim:Name>
            </rim:ExternalIdentifier>
            <rim:ExternalIdentifier registryObject="urn:uuid:d2d7eb19-c5f0-496d-ba47-2be5801a8b41"
identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832" id="ei05"
value="2.16.840.1.113883.3.448">
              <rim:Name>
                <rim:LocalizedString value="XDSSubmissionSet.sourceId"/>
              </rim:Name>
            </rim:ExternalIdentifier>
            <rim:ExternalIdentifier registryObject="urn:uuid:d2d7eb19-c5f0-496d-ba47-2be5801a8b41"
identificationScheme="urn:uuid:6b5aea1a-874d-4603-a4bc-96a0a7b38446" id="ei04"
value="847264406^^^&amp;2.16.840.1.113883.3.448&amp;ISO">
              <rim:Name>
                <rim:LocalizedString value="XDSSubmissionSet.patientId"/>
              </rim:Name>
            </rim:ExternalIdentifier>
          </rim:RegistryPackage>
          <rim:ExtrinsicObject objectType="urn:uuid:7edca82f-054d-47f2-a032-9b2a5b5186c1"
id="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379" mimeType="text/plain">
            <rim:Slot name="URI">
              <rim:ValueList>
                <rim:Value>http://CCDA_Inpatient.xml</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="hash">
              <rim:ValueList>
                <rim:Value>6CD02F982B0BB4A6F23522E10A30930C70DA01BA</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="size">
              <rim:ValueList>
                <rim:Value>104185</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="sourcePatientInfo">
              <rim:ValueList>
                <rim:Value>PID-3|pid1^^^&amp;2.16.840.1.113883.3.448&amp;ISO</rim:Value>
                <rim:Value>PID-5|Doe^John^^^</rim:Value>
                <rim:Value>PID-7|19560527</rim:Value>
                <rim:Value>PID-8|M</rim:Value>
                <rim:Value>PID-11|100 Main St^^Metropolis^Il^44130^USA</rim:Value>
                <rim:Value>PID-13|7325551212</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="sourcePatientId">
              <rim:ValueList>
                <rim:Value>847264406^^^&amp;2.16.840.1.113883.3.448&amp;ISO</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="serviceStartTime">
              <rim:ValueList>
                <rim:Value>20150630144357</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="serviceStopTime">
              <rim:ValueList>
                <rim:Value>20150701134357</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="languageCode">
              <rim:ValueList>
                <rim:Value>en-US</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="creationTime">
              <rim:ValueList>
                <rim:Value>20150701</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Classification classifiedObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379"
classificationScheme="urn:uuid:93606bcf-9494-43ec-9b4e-a7748d1a838d" id="id_11">
              <rim:Slot name="authorSpecialty">
                <rim:ValueList>
                  <rim:Value>Orthopedic</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Slot name="authorRole">
                <rim:ValueList>
                  <rim:Value>Attending</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Slot name="authorInstitution">
                <rim:ValueList>
                  <rim:Value>Cleveland Clinic</rim:Value>
                  <rim:Value>Parma Community</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Slot name="authorPerson">
                <rim:ValueList>
                  <rim:Value>^Smitty^Gerald</rim:Value>
                </rim:ValueList>
              </rim:Slot>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379"
classificationScheme="urn:uuid:93606bcf-9494-43ec-9b4e-a7748d1a838d" id="id_1">
              <rim:Slot name="authorSpecialty">
                <rim:ValueList>
                  <rim:Value>Orthopedic</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Slot name="authorRole">
                <rim:ValueList>
                  <rim:Value>Primary Surgeon</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Slot name="authorInstitution">
                <rim:ValueList>
                  <rim:Value>Cleveland Clinic</rim:Value>
                  <rim:Value>Berea Community</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Slot name="authorPerson">
                <rim:ValueList>
                  <rim:Value>^Dopplemeyer^Sherry</rim:Value>
                </rim:ValueList>
              </rim:Slot>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379"
classificationScheme="urn:uuid:41a5887f-8865-4c09-adf7-e362475b143a"
id="7a6df8d9-a2aa-4df6-ac13-5504457740f8" nodeRepresentation="10160-0">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>2.16.840.1.113883.6.1</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="History and Physical"/>
              </rim:Name>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379"
classificationScheme="urn:uuid:f4f85eac-e6cb-4883-b524-f2705394840f" id="cl03"
nodeRepresentation="1.3.6.1.4.1.21367.2006.7.101">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>Connect-a-thon confidentialityCodes</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="Clinical-Staff"/>
              </rim:Name>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379"
classificationScheme="urn:uuid:a09d5840-386c-46f2-b5ad-9c3699a4309d" id="cl04"
nodeRepresentation="CDAR2/IHE 1.0">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>Connect-a-thon formatCodes</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="CDAR2/IHE 1.0"/>
              </rim:Name>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379"
classificationScheme="urn:uuid:f33fb8ac-18af-42cc-ae0e-ed0b0bdb91e1" id="cl05"
nodeRepresentation="Emergency Department">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>Connect-a-thon healthcareFacilityTypeCodes</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="Emergency Department"/>
              </rim:Name>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379"
classificationScheme="urn:uuid:cccf5598-8b07-4b77-a05e-ae952c785ead" id="cl06"
nodeRepresentation="Emergency">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>Connect-a-thon practiceSettingCodes</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="Emergency"/>
              </rim:Name>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379"
classificationScheme="urn:uuid:f0306f51-975f-434e-a61c-c59651d33983" id="cl07"
nodeRepresentation="34108-1">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>LOINC</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="Outpatient Evaluation and Management"/>
              </rim:Name>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379"
classificationScheme="urn:uuid:2c6b8cb7-8b2a-4051-b291-b1ae6a575ef4" id="cl08"
nodeRepresentation="Colonoscopy">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>Connect-a-thon eventCodeList</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="Colonoscopy"/>
              </rim:Name>
            </rim:Classification>
            <rim:ExternalIdentifier registryObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379"
identificationScheme="urn:uuid:58a6f841-87b3-4a3e-92fd-a8ffeff98427" id="ei01"
value="847264406^^^&amp;2.16.840.1.113883.3.448&amp;ISO">
              <rim:Name>
                <rim:LocalizedString value="XDSDocumentEntry.patientId"/>
              </rim:Name>
            </rim:ExternalIdentifier>
            <rim:ExternalIdentifier registryObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379"
identificationScheme="urn:uuid:2e82c1f6-a085-4c72-9da3-8640a32e42ab" id="ei02"
value="1.3.6.1.4.1.21367.2005.3.9999.32">
              <rim:Name>
                <rim:LocalizedString value="XDSDocumentEntry.uniqueId"/>
              </rim:Name>
            </rim:ExternalIdentifier>
          </rim:ExtrinsicObject>
          <rim:Association
objectType="urn:oasis:names:tc:ebxml-regrep:ObjectType:RegistryObject:Association"
associationType="urn:oasis:names:tc:ebxml-regrep:AssociationType:HasMember"
sourceObject="urn:uuid:d2d7eb19-c5f0-496d-ba47-2be5801a8b41" id="4C3700FC98144F718CCB08657AD83A18"
targetObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379">
            <rim:Slot name="SubmissionSetStatus">
              <rim:ValueList>
                <rim:Value>Original</rim:Value>
              </rim:ValueList>
            </rim:Slot>
          </rim:Association>
          <rim:Classification classificationNode="urn:uuid:a54d6aa5-d40d-43f9-88c5-b4633d873bdd"
classifiedObject="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379"
id="urn:uuid:d2d7eb19-c5f0-496d-ba47-2be5801a8b41"/>
          <rim:ExtrinsicObject objectType="urn:uuid:7edca82f-054d-47f2-a032-9b2a5b5186c1"
id="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3" mimeType="text/plain">
            <rim:Slot name="URI">
              <rim:ValueList>
                <rim:Value>http://Document01.xml</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="hash">
              <rim:ValueList>
                <rim:Value>B9DFE117C13E82C1FDBBCF316A510686F7C21CCD</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="size">
              <rim:ValueList>
                <rim:Value>107485</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="sourcePatientInfo">
              <rim:ValueList>
                <rim:Value>PID-3|pid1^^^&amp;2.16.840.1.113883.3.448&amp;ISO</rim:Value>
                <rim:Value>PID-5|Doe^John^^^</rim:Value>
                <rim:Value>PID-7|19560527</rim:Value>
                <rim:Value>PID-8|M</rim:Value>
                <rim:Value>PID-11|100 Main St^^Metropolis^Il^44130^USA</rim:Value>
                <rim:Value>PID-13|7325551212</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="sourcePatientId">
              <rim:ValueList>
                <rim:Value>847264406^^^&amp;2.16.840.1.113883.3.448&amp;ISO</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="serviceStartTime">
              <rim:ValueList>
                <rim:Value>20150630144357</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="serviceStopTime">
              <rim:ValueList>
                <rim:Value>20150701134357</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="languageCode">
              <rim:ValueList>
                <rim:Value>en-US</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Slot name="creationTime">
              <rim:ValueList>
                <rim:Value>20150701</rim:Value>
              </rim:ValueList>
            </rim:Slot>
            <rim:Classification classifiedObject="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3"
classificationScheme="urn:uuid:93606bcf-9494-43ec-9b4e-a7748d1a838d" id="id_2">
              <rim:Slot name="authorSpecialty">
                <rim:ValueList>
                  <rim:Value>Orthopedic</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Slot name="authorRole">
                <rim:ValueList>
                  <rim:Value>Primary Surgeon</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Slot name="authorInstitution">
                <rim:ValueList>
                  <rim:Value>Cleveland Clinic</rim:Value>
                  <rim:Value>Berea Community</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Slot name="authorPerson">
                <rim:ValueList>
                  <rim:Value>^Dopplemeyer^Sherry</rim:Value>
                </rim:ValueList>
              </rim:Slot>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3"
classificationScheme="urn:uuid:41a5887f-8865-4c09-adf7-e362475b143a"
id="7a6df8d9-a2aa-4df6-ac13-5504457740f9" nodeRepresentation="10160-0">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>2.16.840.1.113883.6.1</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="History and Physical"/>
              </rim:Name>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3"
classificationScheme="urn:uuid:f4f85eac-e6cb-4883-b524-f2705394840f" id="cl13"
nodeRepresentation="1.3.6.1.4.1.21367.2006.7.101">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>Connect-a-thon confidentialityCodes</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="Clinical-Staff"/>
              </rim:Name>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3"
classificationScheme="urn:uuid:a09d5840-386c-46f2-b5ad-9c3699a4309d" id="cl21"
nodeRepresentation="CDAR2/IHE 1.0">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>Connect-a-thon formatCodes</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="CDAR2/IHE 1.0"/>
              </rim:Name>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3"
classificationScheme="urn:uuid:f33fb8ac-18af-42cc-ae0e-ed0b0bdb91e1" id="cl15"
nodeRepresentation="Emergency Department">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>Connect-a-thon healthcareFacilityTypeCodes</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="Emergency Department"/>
              </rim:Name>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3"
classificationScheme="urn:uuid:cccf5598-8b07-4b77-a05e-ae952c785ead" id="cl16"
nodeRepresentation="Emergency">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>Connect-a-thon practiceSettingCodes</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="Emergency"/>
              </rim:Name>
            </rim:Classification>
            <rim:Classification classifiedObject="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3"
classificationScheme="urn:uuid:f0306f51-975f-434e-a61c-c59651d33983" id="cl17"
nodeRepresentation="34108-1">
              <rim:Slot name="codingScheme">
                <rim:ValueList>
                  <rim:Value>LOINC</rim:Value>
                </rim:ValueList>
              </rim:Slot>
              <rim:Name>
                <rim:LocalizedString value="Outpatient Evaluation and Management"/>
              </rim:Name>
            </rim:Classification>
            <rim:ExternalIdentifier registryObject="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3"
identificationScheme="urn:uuid:58a6f841-87b3-4a3e-92fd-a8ffeff98427" id="ei11"
value="847264406^^^&amp;2.16.840.1.113883.3.448&amp;ISO">
              <rim:Name>
                <rim:LocalizedString value="XDSDocumentEntry.patientId"/>
              </rim:Name>
            </rim:ExternalIdentifier>
            <rim:ExternalIdentifier registryObject="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3"
identificationScheme="urn:uuid:2e82c1f6-a085-4c72-9da3-8640a32e42ab" id="ei19"
value="1.3.6.1.4.1.21367.2005.3.9999.33">
              <rim:Name>
                <rim:LocalizedString value="XDSDocumentEntry.uniqueId"/>
              </rim:Name>
            </rim:ExternalIdentifier>
          </rim:ExtrinsicObject>
          <rim:Association
objectType="urn:oasis:names:tc:ebxml-regrep:ObjectType:RegistryObject:Association"
associationType="urn:oasis:names:tc:ebxml-regrep:AssociationType:HasMember"
sourceObject="urn:uuid:d2d7eb19-c5f0-496d-ba47-2be5801a8b41" id="4E7ADAD9D66A406083FE1CC96142D107"
targetObject="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3">
            <rim:Slot name="SubmissionSetStatus">
              <rim:ValueList>
                <rim:Value>Original</rim:Value>
              </rim:ValueList>
            </rim:Slot>
          </rim:Association>
          <rim:Classification classificationNode="urn:uuid:a54d6aa5-d40d-43f9-88c5-b4633d873bdd"
classifiedObject="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3"
id="urn:uuid:d2d7eb19-c5f0-496d-ba47-2be5801a8b41"/>
        </rim:RegistryObjectList>
      </lcm:SubmitObjectsRequest>
      <xdsb:Document id="urn:uuid:dd6d5f7a-1d55-4ca6-bbf8-1a6d0a41a0d3">
        <xop:Include href="cid:C443425E029444B58B50B8F3DF5271A0"
xmlns:xop="http://www.w3.org/2004/08/xop/include"/>
      </xdsb:Document>
      <xdsb:Document id="urn:uuid:a6f8e78f-6c7d-4386-972a-35525f060379">
        <xop:Include href="cid:67BF7E72B0ED4F79987331A72C579A60"
xmlns:xop="http://www.w3.org/2004/08/xop/include"/>
      </xdsb:Document>
    </xdsb:ProvideAndRegisterDocumentSetRequest>
  </soapenv:Body>
</soapenv:Envelope>






--A55D8908BAE949D88C1C22D5B47C6C04
Content-Type: text/plain
Content-Transfer-Encoding: binary
Content-ID: <67BF7E72B0ED4F79987331A72C579A60>






--A55D8908BAE949D88C1C22D5B47C6C04
Content-Type: text/plain
Content-Transfer-Encoding: binary
Content-ID: <C443425E029444B58B50B8F3DF5271A0>


--A55D8908BAE949D88C1C22D5B47C6C04--

Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdAttachmentMemory...MTOM/XOP [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 1, 2015 11:35 AM   in response to: David Keith in response to: David Keith
David wrote:

In building the content for transmission I did the entire MTOM/XOP
thing and attached two documents inside of MIME parts. When I pulled
out the content from TIdLogStream, the MIME attachments aren't there.
The MIME sections are there, but they do not contain the documents.

OK, so what does your code look like that is creating those attachments and
encoding them to the TStream that you are posting to the server? Sounds
like you are missing the encoding step for those attachment parts. Remember,
TIdMessageParts and TIdText/TIdAttachment are just containers, they don't
do the actual encoding for you. That is typically handled by TIdMessageClient
and TIdMultipartFormDataStream, but you are having to bypass them in this
situation, so you have to handle the encoding logic manually in your own
code. TIdAttachment has a SaveToStream() method, if you want to just copy
the attachment data as-is into your TStream ("Content-Transfer-Encoding:
8bit" or "Content-Transfer-Encoding: binary"), or you can use the TIdAttachment.OpenLoadStream()
method to get a TStream that you can pass to TIdEncoderMIME::EncodeStream()
("Content-Transfer-Encoding: base64").

--
Remy Lebeau (TeamB)
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 1, 2015 2:03 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:
David wrote:

In building the content for transmission I did the entire MTOM/XOP
thing and attached two documents inside of MIME parts. When I pulled
out the content from TIdLogStream, the MIME attachments aren't there.
The MIME sections are there, but they do not contain the documents.

OK, so what does your code look like that is creating those attachments and
encoding them to the TStream that you are posting to the server? Sounds
like you are missing the encoding step for those attachment parts. Remember,
TIdMessageParts and TIdText/TIdAttachment are just containers, they don't
do the actual encoding for you. That is typically handled by TIdMessageClient
and TIdMultipartFormDataStream, but you are having to bypass them in this
situation, so you have to handle the encoding logic manually in your own
code. TIdAttachment has a SaveToStream() method, if you want to just copy
the attachment data as-is into your TStream ("Content-Transfer-Encoding:
8bit" or "Content-Transfer-Encoding: binary"), or you can use the TIdAttachment.OpenLoadStream()
method to get a TStream that you can pass to TIdEncoderMIME::EncodeStream()
("Content-Transfer-Encoding: base64").

But Remy that's just it I have to put them into the content sections 'as is' - unencoded for the MTOM/XOP spec or 'binary'; no encoding allowed.

Also when I step through I see the content is assembled properly. Is TIdHTTP stripping out the content of the MIME attachment sections because the content is not encoded??

Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdAttachmentMemory...MTOM/XOP [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 1, 2015 5:01 PM   in response to: David Keith in response to: David Keith
David wrote:

But Remy that's just it I have to put them into the content sections
'as is' - unencoded for the MTOM/XOP spec or 'binary'; no encoding
allowed.

Which is fine. You can use TIdAttachment.SaveToStream() for that purpose.
Are you doing ANYTHING to copy the TIdAttachment data into your posted TStream?
It does not sound like you are, but you have not shown that piece of code
yet.

Also when I step through I see the content is assembled properly.

Not according to what you have shown.

Is TIdHTTP stripping out the content of the MIME attachment
sections because the content is not encoded??

No. The whole purpose of posting a TStream is that TIdHTTP sends the stream
content as-is without any interpretation at all. So it is your responsibility
to make sure the TStream content is accurate before you post it. And you
can verify that by saving the TStream to a file before you post it.

--
Remy Lebeau (TeamB)
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2015 9:33 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:

Which is fine. You can use TIdAttachment.SaveToStream() for that purpose.
Are you doing ANYTHING to copy the TIdAttachment data into your posted TStream?
It does not sound like you are, but you have not shown that piece of code
yet.

What I'm doing is building everything as a string. With two message attachments it comes out to over 6000 lines of text, that's why I've been hesitant to show it here.

So once the text of the message is built as a giant ansi string, here's what I do with it:

// the following code snippets build the message
 
  FHTTP.Request.UserAgent := KICMSUserAgent;
  FHTTP.Request.ContentType := BuildHTTPContentType;
...
 
function TMTOMXOPXDSPackage.BuildHTTPContentType: AnsiString;
begin
  FMainContentId := GetMIMEContentID;
  Result := '' + CONTENTTYPE_MULTIPART_RELATED +
            '; boundary="' + GetMIMEBoundary +
            '"; type="' + CONTENTTYPE_APPLICATION_XOP_XML +
            '"; start="<'>"; start-info="' + CONTENTTYPE_APPLICATION_SOAP_XML +
            '"; action="urn:ihe:iti:2007:' + KXDSAction[FXDSAction] + '"' +
            '';
end;
 
 
  Result := BuildMIMESectionMain;
 
...
 
 
function TMTOMXOPXDSPackage.BuildMIMESectionMain: AnsiString;
begin
  // builds first MIME boundary marker with attributes
  Result := '--' + FMIMEBoundary + #13#10;
  Result := Result + KMIMEAttribContentTypeMain + #13#10;
  Result := Result + KMIMEAttribContentTransferEncoding + #13#10;
  Result := Result + AnsiString(StringReplace(KMIMEAttribContentID,'^id^',FMainContentId,[]) + #13#10#13#10);
end;
 
...
 
  Result := Result + CreateSOAPEnvelope + #13#10#13#10;
 
...
 
function TMTOMXOPXDSPackage.CreateSoapEnvelope: AnsiString;
var
  nx: TNativeXML;
begin
  nx := TNativeXML.Create;
  try
    Result := '<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="http://www.w3.org/2005/08/addressing">' + #13;
    Result := Result + CreateSoapHeader + #13;
    Result := Result + CreateSoapBody + #13;
    Result := Result + '</soapenv:Envelope>';
    nx.XmlFormat := xfReadable;
    nx.ReadFromString(Result);
    Result := nx.WriteToString;
  finally
    FreeAndNil(nx);
  end;
end;
 
...
 
 
  Result := AnsiString(StringReplace(Result,'<?xml version="1.0" encoding="UTF-8"?>',
                                            '<!--********************Start of Metadata********************-->',[]));
  nodelist := TXMLNodeList.Create;
  try
    // Get a nodelist of Document nodes
    FXDSTrans.Root.NodesByName('xdsb:Document',nodeList);
    i := nodeList.Count;
    // build MIME content sections for each document
    for i := 0 to Pred(nodeList.Count) do
      Result := Result + #13#10#13#10# + BuildMIMESectionDoc(FXDSTrans.Documents[i].MIMEContentId);
 
...
 
function TMTOMXOPXDSPackage.BuildMIMESectionDoc(const DocContentId: AnsiString): AnsiString;
var
  tsl: TStringList;
begin
  // builds MIME document section boundary with attributes and closing boundary marker
  tsl := TStringList.Create;
  try
    // loads a document from a previously stored stream
    tsl.LoadFromStream(Transaction.DocStreams[DocContentId]);
    // build MIME content header
    Result := '--' + FMIMEBoundary + #13#10;
    Result := Result + 'Content-Type: ' + CONTENTTYPE_TEXT_PLAIN + #13#10;
    Result := Result + KMIMEAttribContentTransferEncoding + #13#10;
    Result := Result + AnsiString(StringReplace(KMIMEAttribContentID,'^id^',DocContentId,[]) + #13#10#13#10);
    // insert document text into MIME content section
    Result := Result + tsl.Text + #13#10;
  finally
    tsl.Free;
  end;
end;
 
...
 
    // the following line closes MIME attachemnts - only one instance
    // can exist at the end of the last MIME attachment
    Result := Result + '--' + FMIMEBoundary + '--';
  finally
    FXDSbMessage := Result;
    nodeList.Free;
  end;
 
 
// the following code snippet sends the http message to the server
 
procedure TMTOMXOPXDSPackage.SendXDSMessage;
var
  ms,ms2: TMemoryStream;
  tsl: TStringList;
//  rio: THTTPRIO;
//  wsXDS: DocumentRepository_PortType;
begin
  ms := TMemoryStream.Create;
  ms2 := TMemoryStream.Create;
  tsl := TStringList.Create;
  try
    WriteStringToStream(ms,XDSbMessageText);
    ms.Position := 0;
    FHTTPLog.SendStream := ms2;
    FHTTPLog.ReplaceCRLF := False;
    FHTTPLog.Active := True;
    FXDSbResponse := AnsiString(FHTTP.Post(FHTTP.Request.Host,ms));
    ParseSOAPResponse;
    ms2.Position := 0;
    tsl.LoadFromStream(ms2);
    tsl.SaveToFile('HTTPStream.txt');
    FHTTPLog.Active := False;
  finally
    FreeAndNil(tsl);
    FreeAndNil(ms);
    FreeAndNil(ms2);
  end;



No. The whole purpose of posting a TStream is that TIdHTTP sends the stream
content as-is without any interpretation at all. So it is your responsibility
to make sure the TStream content is accurate before you post it. And you
can verify that by saving the TStream to a file before you post it.

I save the content to a TMemo. I then save it to file and run it through the site's validator, and it validates.

I did find one code issue which I have since fixed. It's still failing, but I'm not getting the same error. More details to follow...


--

Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdAttachmentMemory...MTOM/XOP [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2015 2:28 PM   in response to: David Keith in response to: David Keith
David wrote:

What I'm doing is building everything as a string.

Ouch. Why an AnsiString, and not a TStream, like TMemoryStream? You are
already posting the data to the server using TMemoryStream anyway, so I would
strongly suggest you eliminate your AnsiString entirely and just store your
data directly to that TMemoryStream. This is important considering that
your MIME attachment data is binary, and AnsiString is not a suitable container
for binary data in Delphi 2009+. The code you have shown is worrysome because
you are passing the constructed AnsiString to Indy's WriteStringToStream()
function, which is going to perform a charset conversion on the string data,
which is going to mess up your binary attachment data.

With two message attachments it comes out to over 6000 lines of
text, that's why I've been hesitant to show it here.

You could show truncated versions of each piece's data, just to show they
exist. But I'm more interested in seeing the actual code that is building
up the string before you post it to the server. Because that code has a hole
in it that is causing the attachment data to be missing in your output.

function TMTOMXOPXDSPackage.BuildMIMESectionDoc(const DocContentId:
AnsiString): AnsiString;

This is where your attachment data is going missing.

Your earlier messages made it sound like you were using TIdMessageParts and
TIdAttachment instead, but that is clearly not the case.

You are actually loading the attachment data from a TStream returned by Transaction.DocStreams[],
using TStringList.LoadFromStream() and then TStringList.Text to copy the
attachment data into your output string.

Based on the output you showed earlier, clearly the TStringList.Text is empty,
which likely means DocStreams[] is returning a TStream whose Position is
equal to its Size (versus the RTL failing to convert the raw stream bytes
to a viable String when populating the TStringList). So either that TStream
is empty to begin with, or it has been seeked to the end and you are not
seeking it back to the beginning before calling TStringList.LoadFromStream().

Perhaps your earlier code that prepares the Transaction is writing data to
each DocStream but forgetting to reset their Position afterwards.

Try this:

function TMTOMXOPXDSPackage.BuildMIMESectionDoc(const DocContentId: AnsiString): 
AnsiString;
var
  tsl: TStringList;
  stream: TStream;
begin
  // builds MIME document section boundary with attributes and closing boundary 
marker
  tsl := TStringList.Create;
  try
    // loads a document from a previously stored stream
    stream := Transaction.DocStreams[DocContentId];
    stream.Position := 0; // <-- add this
    tsl.LoadFromStream(stream);
    // build MIME content header
    Result := '--' + FMIMEBoundary + #13#10;
    Result := Result + 'Content-Type: ' + CONTENTTYPE_TEXT_PLAIN + #13#10;
    Result := Result + KMIMEAttribContentTransferEncoding + #13#10;
    Result := Result + AnsiString(StringReplace(KMIMEAttribContentID,'^id^',DocContentId,[]) 
+ #13#10#13#10);
    // insert document text into MIME content section
    Result := Result + tsl.Text + #13#10;
  finally
    tsl.Free;
  end;
end;

Now, with that said, I would strongly recomend re-writing this code to eliminate
the AnsiString intermediary when building the post stream. Try something
like this:

function TMTOMXOPXDSPackage.BuildHTTPContentType: AnsiString;
begin
  FMainContentId := GetMIMEContentID;
  Result := Format(
    '%s; boundary="%s"; type="%s"; start="<%s>"; start-info="%s"; action="urn:ihe:iti:2007:%s"',
    [CONTENTTYPE_MULTIPART_RELATED, GetMIMEBoundary, CONTENTTYPE_APPLICATION_XOP_XML, 
FMainContentID, CONTENTTYPE_APPLICATION_SOAP_XML, KXDSAction[FXDSAction]]
  );
end;
 
procedure TMTOMXOPXDSPackage.BuildMIMESectionMain(Output: TStream);
begin
  // builds first MIME boundary marker with attributes
  WriteStringToStream(Output, '--' + FMIMEBoundary + #13#10);
  WriteStringToStream(Output, KMIMEAttribContentTypeMain + #13#10);
  WriteStringToStream(Output, KMIMEAttribContentTransferEncoding + #13#10);
  WriteStringToStream(Output, StringReplace(KMIMEAttribContentID,'^id^',FMainContentId,[]) 
+ #13#10#13#10);
end;
 
procedure TMTOMXOPXDSPackage.CreateSoapEnvelope(Output: TStream);
var
  xml: String;
  nx: TNativeXML;
begin
  nx := TNativeXML.Create;
  try
    xml := '<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" 
xmlns:wsa="http://www.w3.org/2005/08/addressing">' + #13;
    xml := xml + CreateSoapHeader + #13;
    xml := xml + StringReplace(CreateSoapBody, '<?xml version="1.0" encoding="UTF-8"?>', 
'<!--********************Start of Metadata********************-->', []) + 
#13;
    xml := xml + '</soapenv:Envelope>';
    nx.XmlFormat := xfReadable;
    nx.ReadFromString(xml);
    WriteStringToStream(Output, nx.WriteToString); // does TNativeXml have 
a WriteToStream() method?
  finally
    FreeAndNil(nx);
  end;
end;
 
procedure TMTOMXOPXDSPackage.BuildMIMESectionDoc(const DocContentId: String; 
Output: TStream);
begin
  // builds MIME document section boundary with attributes and closing boundary 
marker
  // build MIME content header
  WriteStringToStream(Output, '--' + FMIMEBoundary + #13#10);
  WriteStringToStream('Content-Type: ' + CONTENTTYPE_TEXT_PLAIN + #13#10);
  WriteStringToStream(KMIMEAttribContentTransferEncoding + #13#10);
  WriteStringToStream(StringReplace(KMIMEAttribContentID,'^id^',DocContentId,[]) 
+ #13#10#13#10);
  // insert document text into MIME content section
  Output.CopyFrom(Transaction.DocStreams[DocContentId], 0);
  WriteStringToStream(Output, #13#10);
end;
 
procedure TMTOMXOPXDSPackage.BuildXDSbMessage(Output: TStream);
var
  nodelist: TXMLNodeList;
  i: Integer;
begin
  BuildMIMESectionMain(Output);
  CreateSOAPEnvelope(Output);
  WriteStringToStream(Output, #13#10#13#10);
  nodelist := TXMLNodeList.Create;
  try
    // Get a nodelist of Document nodes
    FXDSTrans.Root.NodesByName('xdsb:Document', nodeList);
    // build MIME content sections for each document
    for i := 0 to Pred(nodeList.Count) do
      BuildMIMESectionDoc(FXDSTrans.Documents[i].MIMEContentId, Output);
  finally
    nodeList.Free;
  end;
  // the following line closes MIME attachemnts - only one instance
  // can exist at the end of the last MIME attachment
  WriteStringToStream(Output, '--' + FMIMEBoundary + '--');
end;
 
procedure TMTOMXOPXDSPackage.SendXDSMessage;
var
  msg, log: TMemoryStream;
begin
  log := TMemoryStream.Create;
  try
    msg := TMemoryStream.Create;
    try
      BuildXDSbMessage(msg);
      msg.Position := 0;
 
      FHTTP.Request.UserAgent := KICMSUserAgent;
      FHTTP.Request.ContentType := BuildHTTPContentType;
 
      FHTTPLog.SendStream := log;
      FHTTPLog.ReplaceCRLF := False;
      FHTTPLog.Active := True;
      try
        FXDSbResponse := FHTTP.Post(FHTTP.Request.Host, msg);
      finally
        FHTTPLog.Active := False;
      end;
    finally
      msg.Free;
    end;
    ParseSOAPResponse;
    log.SaveToFile('HTTPStream.txt');
  finally
    log.Free;
  end;
end;


--
Remy Lebeau (TeamB)
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: TIdAttachmentMemory...MTOM/XOP [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2015 4:09 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy,

| Try this:

You are positively amazing!!! KUDOS!

--

Q -- XanaNews 1.19.1.372 - 2015-07-02 16:08:39
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: TIdAttachmentMemory...MTOM/XOP [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2015 4:48 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy,

| Try this:

You are positively amazing!!! KUDOS!

--

Q -- XanaNews 1.19.1.372 - 2015-07-02 16:08:39
Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: TIdAttachmentMemory...MTOM/XOP [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 6, 2015 10:13 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy,

| Try this:

You are positively amazing!!! KUDOS!

--

Q -- XanaNews 1.19.1.372 - 2015-07-02 16:08:39
David Keith

Posts: 196
Registered: 12/10/99
Re: TIdAttachmentMemory...MTOM/XOP [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 8, 2015 10:19 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy Lebeau (TeamB) wrote:

David wrote:

What I'm doing is building everything as a string.

Ouch. Why an AnsiString, and not a TStream, like TMemoryStream? You are
already posting the data to the server using TMemoryStream anyway, so I would
strongly suggest you eliminate your AnsiString entirely and just store your
data directly to that TMemoryStream. This is important considering that
your MIME attachment data is binary, and AnsiString is not a suitable container
for binary data in Delphi 2009+. The code you have shown is worrysome because
you are passing the constructed AnsiString to Indy's WriteStringToStream()
function, which is going to perform a charset conversion on the string data,
which is going to mess up your binary attachment data.

With two message attachments it comes out to over 6000 lines of
text, that's why I've been hesitant to show it here.

You could show truncated versions of each piece's data, just to show they
exist. But I'm more interested in seeing the actual code that is building
up the string before you post it to the server. Because that code has a hole
in it that is causing the attachment data to be missing in your output.

This is where your attachment data is going missing.

Your earlier messages made it sound like you were using TIdMessageParts and
TIdAttachment instead, but that is clearly not the case.

You are actually loading the attachment data from a TStream returned by Transaction.DocStreams[],
using TStringList.LoadFromStream() and then TStringList.Text to copy the
attachment data into your output string.

Based on the output you showed earlier, clearly the TStringList.Text is empty,
which likely means DocStreams[] is returning a TStream whose Position is
equal to its Size (versus the RTL failing to convert the raw stream bytes
to a viable String when populating the TStringList). So either that TStream
is empty to begin with, or it has been seeked to the end and you are not
seeking it back to the beginning before calling TStringList.LoadFromStream().

Perhaps your earlier code that prepares the Transaction is writing data to
each DocStream but forgetting to reset their Position afterwards.

Try this:


Now, with that said, I would strongly recomend re-writing this code to eliminate
the AnsiString intermediary when building the post stream. Try something
like this:

I tried to post last week but the forums were down (again). I did find the missing stream position
call and fixed it. This in turn gave me the complete message built in my AnsiString. I then began
submitting the message to the server and getting back valid parse error messages.

One of the most frustrating things I found is that the server that I am sending to always gets a
different size and hash of the message than the size and hash that I get. I get my size and hash by
loading the MIME content - in this case XML - into a stream and reading the size.

The sizes I get from the stream always read as too large on one document and too small on the
other. Since the document has to be placed inside of a MIME section, CR/LF's come into play. So I
eliminated all CR's since the host platform is linux, which got me within 1 byte of the server's
size of the documents. One document is 1 byte too small, the other document is 1 byte too large.

Of course because we differ in sizes we also differ in hashes as well. Since the size and hash
nodes are supposed to be optional, I removed them and resubmitted the message. Then I get a strange
error from the server that the AXIS 2 server admins have so far not been able to explain. So I'm at
a full stop, unless I can figure out how to shave the extra byte from 1 doc and the missing byte
from the other doc.

Of course this will all run on auto pilot, so whatever is causing the byte counts to be off must be
resolved if this is to work.

Since you have a lot of experience with MIME content I'm hoping you can give advice in relation to
the byte counts.

Thanks again for spelling all of this out. You've enabled me to get much closer to having this new
capability in the bag.


--

Quentin Correll


Posts: 2,412
Registered: 12/1/99
Re: TIdAttachmentMemory...MTOM/XOP [Edit]
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 8, 2015 10:26 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
Remy,

| Try this:

You are positively amazing!!! KUDOS!

--

Q -- XanaNews 1.19.1.372 - 2015-07-02 16:08:39
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: TIdAttachmentMemory...MTOM/XOP
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 1, 2015 11:45 AM   in response to: David Keith in response to: David Keith
David wrote:

Here's a snippet of the first part of the transmission:

That is not really helpful without seeing the actual message body that goes
with it.

Could the '<EOL>'s be a problem?

No, those are simply how Indy's TIdLog... components log CRLFs when the ReplaceCRLF
property is true (which it is by default).

--
Remy Lebeau (TeamB)
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02