Large Payload testing

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Large Payload testing

sc000ter
Hello,We are starting to do large payload testing on CONNECT 4.4.1 running on a GF 3.1.2 server. I have run through all the setups <a href="https://connectopensource.atlassian.net/wiki/display/CONNECTWIKI/Large Payload Testing">here. We are connecting to a Gateway that we do not control that supposedl can accept large file sizes.When we submit to this gateway however, they are seeing "Missing payload" and "Missing XSD document" errors. We typically just spool up the SOAP request and base 64 encode the payload on the fly and call the "submit" of the document submission web service. Does this mean that now we will have to store the base64 encoded payload in the "inbound" directory and then put the base64 encoded filename in the "Document01" element? Thanks, Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

Sailaja
Hi,

Yes, you will have to store payload file in "inbound" directory and provide a base64 encoded path in Document Submission service for large payload.

Follow Document Submission (Deferred) execution steps from wiki : https://connectopensource.atlassian.net/wiki/display/CONNECTWIKI/Large+Payload+Testing

Hope this will resolve issue.

Thanks
Sailaja Adusumilli
CONNECT Product Team
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

sc000ter
Looks like I got the issue resolved.  I missed setting the ParsePayloadAsFileURIOutbound to true.  I couldn't find it in the gateway.properties so I added it and set it to true but then lower down I had it again and it was set to false.

Once I corrected that the target gateway started seeing my submissions.

Thanks for the help!
Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

sc000ter
In reply to this post by Sailaja
I spoke too soon.

The receiving gateway is still reporting "XDS MISSING DOCUMENT" from my submissions.

However, they can compute the file size.

I did set the webserviceproxy.timeout to zero in the gateway.properties file.

Is there anything else I can check or additional debugging I can do to track this down?

Thanks,
Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

Sailaja
Hi Brian,

Looks like your configuration for webserviceproxy.timeout property is not correct. As per largepayload wiki instructions this should at least set to 480000 milliseconds.

Please follow Gateway Setup -> Common Setup instructions from wiki : https://connectopensource.atlassian.net/wiki/display/CONNECTWIKI/Large+Payload+Testing

Hope this will resolve the issue.

Thanks
Sailaja Adusumilli
CONNECT Product Team
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

sc000ter
Sailaja,

I had a support call yesterday with the people who operate the receiving gateway and they instructed me to put this value at zero.

Everything else is setup as per the web page you keep linking me to.

Thanks,
Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

Sailaja
Hi Brian,

I thought you might have overlooked webserviceproxy.timeout property and set it to "0". Can you please try setting this property as per wiki instructions and see if that helps to resolve issue.

Also, would you mind to share to whom you work with and who is your receiving gateway.

Thanks
Sailaja Adusumilli
CONNECT Product Team
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

sc000ter
Sailaja,

Thanks for your reply.

I set the web service proxy timeout back to 480000 and tried another submission.  

I get this in my log file:

[#|2016-09-01T10:10:20.516-0500|INFO|glassfish3.1.2|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=124;_ThreadName=Thread-2;|[01/10:10:20:516] INFO   Log4jEventLogger                END_INBOUND_MESSAGE has triggered. It has messageID urn:uuid:a7ff0860-ec34-4342-87da-5c7875c85e75, transactionID null and description {"initiating_hcid":"2.16.840.1.113883.13.34.110.2","error_codes":[],"responding_hcids":["urn:oid:2.16.840.1.113883.3.1066.2"],"service_type":"Document Submission Deferred Response","action":"2.0","response_ids":["urn:uuid:3518c90a-1e51-4ed9-a2fd-b9ac9c36653a","urn:uuid:1888445b-6bdc-4e82-b024-0224bc9fcff1","urn:uuid:9c70e1c7-3816-42e2-8af0-ca1464bebb34"],"message_id":"urn:uuid:a7ff0860-ec34-4342-87da-5c7875c85e75","statuses":["urn:oasis:names:tc:ebxml-regrep:ResponseStatusType:ResponseAccepted"]}

But no corrosponding SOAP message.

I am working with esMD.

Thanks,
Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

Sailaja
Hi Brian,

Loginfo shows that you executed DS Deferred Response testcase and transaction completed without any error codes. I believe your SOAP test passed.

Can you please provide your payload, server.log info to trace down further.

Thanks


Sailaja Adusumilli
CONNECT Product Team
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

sc000ter
Sailaja,

Sure, I will upload my server.log file.

As for the payload, I'll have to research that one.  We use a .NET windows service with a service reference to our gateway's Defferred Document Submission and I'm not sure how to save the payload from that.

server.log

I will see if I can get the payload and upload it into another post.

Thanks,
Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

Minh
Administrator
Hi Brian
After looking at the log, it turns out the payload did not convert correct.  It should be in base64 encode format from file:///<file path location> . For example:
your encode value should be simliar like this ZmlsZTovLy9jOi9sYXJnZXBheWxvYWQvb3V0Ym91bmQvdGVzdC50eHQ=

You can find more information under "Gateway setup->1c" https://connectopensource.atlassian.net/wiki/x/KwB3AQ.

Hope this will help you out.

Thanks
Minh-Hai Nguyen
CONNECT Product Team Member
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

sc000ter
Thank you for your quick replies! :)

When I put together the file path and base 64 encode it, it is thus:
ZmlsZTovLy9DOi9MYXJnZVBheWxvYWRGaWxlcy9vdXRib3VuZC8wNTBkNDliZC0wODcxLTRhOGQtYjY5Zi00MDY1ZTIyZjBhNDkudHh0

When I decode it using base64decode.org, It is my correct file path:
file:///C:/LargePayloadFiles/outbound/050d49bd-0871-4a8d-b69f-4065e22f0a49.txt

We call our gateway from .NET and the ProvideAndRegisterDocumentSetRequestTypeDocument.Value property wants a byte array.  I am using Convert.FromBase64String to put in the byte array.

Thanks,
Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

Minh
Administrator
Hi Brian,

I just want to see where it fails.  Please correct me if my assumption is wrong.
Initiator Gateway (Java): glassfish
Responding Gateway (.NET)

Does your responding gateway(.net) receive glassfish gateway(java) inbound message correct?  If yes, then the responding gateway need to convert byte array into whatever it requires since the WSDL schema indicate byte array.  In our java side, we implement mtom to transfer big file over wire.  Maybe the .NET gateway should have some options to enable MTOM in order to receive it.

Hope this help.
Minh-Hai Nguyen
CONNECT Product Team Member
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

sc000ter
Hi Minh,

You have it backwards.

.NET is calling initiating GlassFish Gateway, which in turn, is calling out to the receiving GlassFish gateway.

I suppose you could say that we are using .NET as a proxy. to call the DocumentSubmissionDefferredRequestUnsecured.  

Does this make sense?

I do have MTOM configured in .NET.
 <customBinding>
                <binding name="EntityXDRAsyncRequest_Binding" sendTimeout="00:10:00" closeTimeout="00:10:00" openTimeout="00:10:00">
                  <mtomMessageEncoding messageVersion="Soap12" writeEncoding="utf-8" />
                 
                  <httpTransport />
                </binding>
            </customBinding>

Thanks,
Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

sc000ter
In reply to this post by Minh
One more question here.

Does the physical file need to be a base64 encoded text file?  

Or is it the actual payload unencoded?

Thanks,
Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

Minh
Administrator
The physical file doesn't need to be base64 encode text file.  The payload need to be base64 file uri encode path so that initiator gateway can convert into datahandler (in java).  I don't know how .NET do in term of conversion.  We tested locally from glassfish (initiator) to wildfly (responding) gateway and could not duplicate the issue.  To isolate the problem, I would recommend to use soapUI call initiator glassfish which in turn calls your responding glassfish (SoapUI->glassfish->glassfish instead of .NET->Glassfish->glassfish) to test.  
Here is a link to see how we convert in java in method convertFileLocationToDataIfEnabled(https://github.com/CONNECT-Solution/CONNECT/blob/432ed8a6b301d937dc0adecc54bb387d639b9a1e/Product/Production/Services/DocumentSubmissionCore/src/main/java/gov/hhs/fha/nhinc/docsubmission/DocSubmissionUtils.java).  I hope you can find equivalent in .NET
Minh-Hai Nguyen
CONNECT Product Team Member
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

sc000ter
Minh,

Things seem to work if I write the payload as unencoded XML to the hard-drive.

However, it seems that I am unable to delete the file after I make the submission.  Java seems to have a handle on the file and I am unable to delete it from the file system.

Thanks,
Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

Minh
Administrator
Hi Brian,

There are probably some weird things happen at FileStream handler on glassfish server.  Can you restart the server and see if you can delete it? Also if possible, can you monitor and keep track how often this issue comes up?

Thanks,
Minh-Hai Nguyen
CONNECT Product Team Member
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

sc000ter
Minh,

This happens all the time.

Whenever I configure for large payload, the files I write to disk can not be deleted until I restart GlassFish.

I can't have that.  We do hundreds of submissions a day.

How can I troubleshoot and fix this?

Thanks,
Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large Payload testing

sc000ter
In reply to this post by Minh
Minh,

I have another question related to large payload testing as well.  It has to do with the SOAP messages that are returned to us from the receiving gateway.  When I configure GF/CONNECt for large payload, no SOAP messages are written to the log files or the MySQL database.

Is there a way to turn SOAP logging back on to the log files at least?  We sweep the log files and archive them off the box to get our responses so I'm not too worried about server log file sizes.

Thanks,
Brian
12
Loading...