SP V3 metadata types

[Brent Putman]

On 6/13/18 11:20 PM, Cantor, Scott wrote:
> On 6/13/18, 9:33 AM, "dev on behalf of Tom Scavo" <dev-bounces at shibboleth.net on behalf of trscavo at gmail.com> wrote:
>
>> Right, that agrees with what Scott and Rod told me earlier, but that
>> is suboptimal since the metadata resolver redundantly processes the
>> metadata in the case of 304. That processing includes signature
>> verification and who knows whet other metadata filters have been
>> configured on the DynamicHTTPMetadataProvider. I'm not intimately
>> familiar with the implementation but there seems to be a significant
>> optimization lurking here, essentially the same optimization
>> implemented in FileBackedHTTPMetadataProvider.

Well, this was a conscious choice and not a bug or oversight.  We
deliberately decided to move all the conditional GET processing into
the caching HttpClient, to eliminate the complexity of the resolver
code.  So I guess one could say that we chose to optimize the software
components for separation-of-concerns and simplicity, vs. runtime
computational cost.

I personally do not see the additional overhead of the latter as being
significant enough to revisit that decision, but if others do, then we
can.  It would very possibly mean effectively abandoning entirely the
use of caching HttpClients across the system, since that's the whole
reason we have them (it also supports Cache-Control-based local
caching, but the same argument about re-parsing and re-processing
applies I think).


> Yes, exactly, that's one of the enhancements I asked for, the other being caching failure to avoid repeated failed lookups.


Actually, I don't recall that you have asked for this (resolver to
implement condition GET internally).  That is not what I got from
OSJ-234 (which is about 1) how we compute the next refresh for the edge
case of the source returning null, and 2) caching negative lookups).

In fact, IIRC I think we have in the past discussed explicitly the
re-processing implications of use of the caching HttpClient, and I
thought we were fine with it. 

>
> The same thing happens with LocalDynamic. Even if the local file is unchanged, it will reload it every time it tries, and go through the full processing step when it does.

Yes, for local file load I did understand that and was working on
enhancements to XMLObjectLoadSaveManager.  That really is just an
internal tweak.

Switching to internal eval of conditional GET for the HTTP resolvers
would likely not be a tweak, it would pretty much be a redesign.  The
scope kind of depends on whether we can get a caching HttpClient to
accept and pass-through the callers conditional GET headers and return
the server's 304 response rather than the previously cached response. 
I am not sure if it can do that, I would have to research.  If not,
then we'd be looking at a significant change, and all the stuff we
expose everywhere for caching HttpClients would become mostly or
entirely moot.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/dev/attachments/20180614/c473f669/attachment.html>