<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 8/18/20 8:34 AM, Cantor, Scott
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:1769A01A-6E63-4884-A9F4-4D82981E6313@osu.edu">
<pre class="moz-quote-pre" wrap="">On 8/18/20, 12:59 AM, "users on behalf of Paul B. Henson" <a class="moz-txt-link-rfc2396E" href="mailto:users-bounces@shibboleth.netonbehalfofhenson@cpp.edu"><users-bounces@shibboleth.net on behalf of henson@cpp.edu></a> wrote:
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap=""> I dunno, after killing most of the day trying to track this down I think
I might be stalled. Any thoughts on where to dig further?
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
No, Brent has to look at it. But if we can't reproduce it, we might have to come back and try and set up a test against the system that's failing.
</pre>
</blockquote>
<p><br>
</p>
<p>Paul, thanks for all the legwork and detailed logging. That
pretty much confirms *what* is happening. The
ThreadLocalX509TrustManager which actually calls the trust engine
and sets the server TLS validation state is not being called in
the subsequent requests' failure cases. So the suggested cause in
the original error message is misleading - the correct socket
factory IS clearly configured (the thing the canary checks for).
The failure is more literal. The TrustEngine is not being
invoked.</p>
<p>I'm not yet sure *why* it's not being called. That class is our
own impl of the X509TrustManager interface, which is a standard
JSSE component (Java's standard TLS library). So something about
the context in which JSSE is being used here is resulting in
server TLS not being performed in some cases.</p>
<p>As was already discussed, I'd off-hand suspect HttpClient
connection caching. I did not think the HC connection pool cached
the TLSness of the connections, only the TCP socket. But if
that's wrong and it doesn't perform server TLS via JSSE on
subsequent new HTTP requests over a cached connection, then that
sounds like the culprit. I'll have to dig into the details of
HttpClient and confirm.</p>
<p>If that is the issue, then off-hand I'm not sure if there's a
simple workaround. Hopefully there will be a way to adjust the
behavior of the HC PoolingHttpClientConnectionManager to perform
server TLS anew on each request. If not, we'll have to go from
there. (I don't think using their other connection pool impl
BasicHttpClientConnectionManager is appropriate for a server-side
multi-threaded use case.)<br>
</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:1769A01A-6E63-4884-A9F4-4D82981E6313@osu.edu">
<pre class="moz-quote-pre" wrap="">
It does happen that our shibboleth.net IdP is on CentOS 8 and so forth. I have no experience with CAS. Maybe you could help us set up a test case if we enable that server and maybe have you set up the necessary CAS client to point at it for testing?</pre>
</blockquote>
<p><br>
</p>
<p>Since we don't know yet, it's possible it has something
specifically to do with the CAS flow - but at the moment I'd
suspect not. I think it's more likely that this case merely
triggers the bug more readily because it just happens to make
multiple requests to the same route (as defined by HC) in a
relatively short period of time. Something like metadata fetching
wouldn't trigger it I think. MDQ could, but I don't think we
typically use TrustEngine-based server TLS there.</p>
<p>Probably the REST data connector could trigger, if it's calling
the same route for all users (and using TrustEngine TLS). So the
fact that we haven't seen it there is curious, unless people
typically don't do TLS that way with that component.</p>
<p>So I'm not 100% on the exact "why" yet. Will keep investigating.<br>
</p>
<p>--Brent <br>
</p>
</body>
</html>