org.apache.cxf.binding.soap.SoapFault: General security error (Error during certificate path validation: signature check failed ... Caused by java.security.SignatureException: Signature does not match

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

org.apache.cxf.binding.soap.SoapFault: General security error (Error during certificate path validation: signature check failed ... Caused by java.security.SignatureException: Signature does not match

johnhd_at_zen
This post was updated on .
Hi Everyone,

I'm having some trouble apparently verifying the signature on an inbound XCPD from a well-known vendor.

As background, I have successful connections in both directions as part of one of the major Sequoia-managed networks, including TO this vendor.

This means that I'm trusting the intermediate signing-certificate that Sequoia uses (i.e. it's loaded in CA certs). I'm actually trusting TWO of these because the intermediate CA was actually re-keyed recently.

Right now my gateway, an instance of CONNECT 4.4.1, is throwing this error message when I receive their XCPD:

[#|2017-07-10T06:06:21.279-0700|WARNING|glassfish3.1.2|org.apache.cxf.phase.PhaseInterceptorChain|_ThreadID=261;_ThreadName=Thread-2;|Interceptor for {urn:ihe:iti:xcpd:2009}RespondingGateway_Service#{urn:ihe:iti:xcpd:2009}RespondingGateway_PRPA_IN201305UV02 has thrown exception, unwinding now
org.apache.cxf.binding.soap.SoapFault: General security error (Error during certificate path validation: signature check failed)
	at org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor.createSoapFault(WSS4JInInterceptor.java:760)
	at org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor.handleMessage(WSS4JInInterceptor.java:331)
	at org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor.handleMessage(WSS4JInInterceptor.java:93)
	at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:271)
	at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
	at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:239)
	at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:218)
	at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:198)
	at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:137)
	at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:158)
	at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:243)
	at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:163)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:688)
	at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:219)
	at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
	at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)
	at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)
	at com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)
	at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)
	at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)
	at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)
	at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)
	at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)
	at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
	at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
	at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
	at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
	at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
	at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
	at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
	at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
	at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.ws.security.WSSecurityException: General security error (Error during certificate path validation: signature check failed)
	at org.apache.ws.security.components.crypto.Merlin.verifyTrust(Merlin.java:838)
	at org.apache.ws.security.validate.SignatureTrustValidator.verifyTrustInCert(SignatureTrustValidator.java:213)
	at org.apache.ws.security.validate.SignatureTrustValidator.validate(SignatureTrustValidator.java:72)
	at org.apache.ws.security.validate.SamlAssertionValidator.verifySignedAssertion(SamlAssertionValidator.java:121)
	at gov.hhs.fha.nhinc.callback.cxf.CONNECTSamlAssertionValidator.checkSignedAssertion(CONNECTSamlAssertionValidator.java:303)
	at gov.hhs.fha.nhinc.callback.cxf.CONNECTSamlAssertionValidator.validate(CONNECTSamlAssertionValidator.java:284)
	at org.apache.ws.security.processor.SAMLTokenProcessor.handleSAMLToken(SAMLTokenProcessor.java:188)
	at org.apache.ws.security.processor.SAMLTokenProcessor.handleToken(SAMLTokenProcessor.java:78)
	at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:396)
	at org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor.handleMessage(WSS4JInInterceptor.java:274)
	... 36 more
Caused by: java.security.cert.CertPathValidatorException: signature check failed
	at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:159)
	at sun.security.provider.certpath.PKIXCertPathValidator.doValidate(PKIXCertPathValidator.java:347)
	at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:191)
	at java.security.cert.CertPathValidator.validate(CertPathValidator.java:279)
	at org.apache.ws.security.components.crypto.Merlin.verifyTrust(Merlin.java:814)
	... 45 more
Caused by: java.security.SignatureException: Signature does not match.
	at sun.security.x509.X509CertImpl.verify(X509CertImpl.java:451)
	at sun.security.provider.certpath.BasicChecker.verifySignature(BasicChecker.java:160)
	at sun.security.provider.certpath.BasicChecker.check(BasicChecker.java:139)
	at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:133)
	... 49 more
|#]

And we're returning a 500 to them simply stating:

<soap:Text xml:lang="en">General security error (Error during certificate path validation: signature check failed)</soap:Text></soap:Reason>

I've tried the obvious steps including triple-checking that these CA certs are in my cacerts.jks (they are, and my other successful connections attest to the fact that I'm definitely trusting them).

I've even gone so far as to confirm the Authority Key Identifier on the head cert being presented to me in the SAML header:

            X509v3 Authority Key Identifier: 
                keyid:5F:AE:CE:71:93:64:38:6E:DF:76:8F:5F:CB:2D:36:0C:14:30:FF:BC

Matches the Subject Key Identifier of the current PROD Sequioa Entrust CA (which is easily available HERE under the "Display" button on the left)

            X509v3 Subject Key Identifier: 
                5F:AE:CE:71:93:64:38:6E:DF:76:8F:5F:CB:2D:36:0C:14:30:FF:BC
 
The only strange thing at all about their head certificate I've found so far is that, although it's "Validity- Not After:" isn't until early August 2017 (so it's still good for about a month), it has this little-used "Private Key Usage Period" extension with an expiration date that's passed:

X509v3 Private Key Usage Period: 
Not Before: Aug 5 17:10:55 2016 GMT, Not After: Apr 18 05:40:55 2017 GMT

I compared this against the partners I've had success with. While I've been able to CONNECT TO some other partners who have a similarly expired Private Key Usage Period, none of the partners that have successfully (or otherwise) connected TO ME have an expired date.

I'm not very familiar with this extension but some digging revealed that:

The X.509 Certificate and CRL profile presented in RFC 3280 specifies the private key usage period
 extension for allowing the certificate issuer to specify a different validity period for the private key than
 the certificate. This extension is intended for use with digital signature keys. This extension consists of 
two optional components notBefore and notAfter. The private key associated with the certificate should 
not be used to sign objects before or after the times specified by the two

So, if I understand this correctly, while the certificate/keypair is valid for establishing SSL connections, it is no longer valid for digital signatures, which is the use case where we're failing these messages.

I wanted to ask if anyone could shed some light on what's going on in this stacktrace and maybe confirm or deny my findings?
Reply | Threaded
Open this post in threaded view
|

Re: org.apache.cxf.binding.soap.SoapFault: General security error (Error during certificate path validation: signature check failed ... Caused by java.security.SignatureException: Signature does not match

johnhd_at_zen
I've apparently been able to circumvent this issue by importing the head cert (included in the X509Data element of the SAML header) into cacerts.jks.

I don't see this as a solution, but a workaround. It negates a lot of the benefit of Sequoia network connections to have to maintain each partner's head cert, every renewal, so I'm still curious as to the root cause here to see either what's wrong with my gateway or what feedback I can give to the partner and/or Sequoia.
Reply | Threaded
Open this post in threaded view
|

Re: org.apache.cxf.binding.soap.SoapFault: General security error (Error during certificate path validation: signature check failed ... Caused by java.security.SignatureException: Signature does not match

Minh
Administrator
Hi John,
We are currently looking into the certification issue and have some questions for you:
1. "I've apparently been able to circumvent this issue by importing the head cert (included in the X509Data element of the SAML header) into cacerts.jks"-->What serial number of this head cert that you mention here?  Is this "4a a7 f7 19" or "‎4a a7 c2 6d"?  "4a a7 f7 19" is new Intermediate  and "‎4a a7 c2 6d" is the new Root.
2.  In your cacert.jks, you probably have gateway alias which is yr server cert.   Can you export your server cert to see whether it has chain of trust (yr server cert links to Sequoia intermediate(4a a7 f7 19) which links to Root(4a a7 c2 6d)).

Here is the wiki page (https://connectopensource.atlassian.net/wiki/x/z5QUBw) where we will document our finding and test result.

Minh-Hai Nguyen
CONNECT Product Team Member
Reply | Threaded
Open this post in threaded view
|

Re: org.apache.cxf.binding.soap.SoapFault: General security error (Error during certificate path validation: signature check failed ... Caused by java.security.SignatureException: Signature does not match

johnhd_at_zen
Minh wrote
Hi John,
We are currently looking into the certification issue and have some questions for you:
1. "I've apparently been able to circumvent this issue by importing the head cert (included in the X509Data element of the SAML header) into cacerts.jks"-->What serial number of this head cert that you mention here?  Is this "4a a7 f7 19" or "‎4a a7 c2 6d"?  "4a a7 f7 19" is new Intermediate  and "‎4a a7 c2 6d" is the new Root.
2.  In your cacert.jks, you probably have gateway alias which is yr server cert.   Can you export your server cert to see whether it has chain of trust (yr server cert links to Sequoia intermediate(4a a7 f7 19) which links to Root(4a a7 c2 6d)).

Here is the wiki page (https://connectopensource.atlassian.net/wiki/x/z5QUBw) where we will document our finding and test result.
1. Neither. The head certificate is that which the partner itself uses to identify itself, and is signed by (and chains with) either the old or new intermediate Sequoia certificate. I'm able to trust this cert for SSL and/or SAML purposes by trusting ONLY the old root & intermediate and/or new root & intermediate certificates in cacerts.jks in general. This is how it's supposed to work, and it works this way for many partners, but in this particular case I had to import the head cert specifically into my cacert.jks to quell the complaint.

Per my Authority Identifier & Subject Key Identifier discussion above, the head certificate in use is signed by the current Sequoia intermediate, Authority Key Identifier 5F:AE:CE:71:93:64:38:6E:DF:76:8F:5F:CB:2D:36:0C:14:30:FF:BC.

For privacy considerations I can PM you the head certificate itself if you like, but I don't want to leave it here on the forum since it's one of our clients.

2. In cacerts.jks I have both the old & new sequoia root & intermediate certificates. I've confirmed that the Subject Key Identifier of the signing cert on my head cert is the same 5F:AE:CE:71:93:64:38:6E:DF:76:8F:5F:CB:2D:36:0C:14:30:FF:BC. (i.e. issued by the same CA)

I think this addresses your questions? Let me know your thoughts when you can.
Reply | Threaded
Open this post in threaded view
|

Re: org.apache.cxf.binding.soap.SoapFault: General security error (Error during certificate path validation: signature check failed ... Caused by java.security.SignatureException: Signature does not match

johnhd_at_zen
In reply to this post by johnhd_at_zen
I think we have the answer.

We actually ran into this same issue with another client. We worked around it the same way, by adding the partner's head certificate to cacerts.jks.

The similarities between these two certs is that they both were issued by the OLD sequoia intermediate CA, prior to the re-key.

The difference is that this one did not have a"Private Key Usage Period" that was expired.

I heard a rumour from a major vendor that they discovered that CONNECT is unable to trust multiple trust chains. I don't know if this is an artifact of two nearly-identical trust chains because of the intermediate cert re-key mentioned above, or if this means any two trust chains.

Apparently this vendor also encountered this issue in their own solution, which I don't think is an implementation of CONNECT, so it may be a broader Java issue, or something like that.

So the takeaway here is:

1) If you have more than one trust chain in your truststore, especially more than one for the "same" CAs, you might be relegated to importing head certificates

2) There might be an opportunity here to investigate and improve CONNECT's trust scheme
Reply | Threaded
Open this post in threaded view
|

Re: org.apache.cxf.binding.soap.SoapFault: General security error (Error during certificate path validation: signature check failed ... Caused by java.security.SignatureException: Signature does not match

johnhd_at_zen
Final note, a hotfix for this has reportedly been developed and is available to look at here:  https://github.com/CONNECT-Solution/CONNECT/pull/1436