Shibboleth FileBackedHTTPMetadataProvider

[Brent Putman]

On 3/29/16 9:15 AM, Tom Scavo wrote:
> [I'll use Shib IdP V3 syntax below but my question is intended to
> cover both the IdP and SP]

So I'll address the Java/OpenSAML/IdP side.  Sounds like it might
differ from the SP vis-a-vis Scott's other comments.

> Basically, I'm trying to understand the strategy around the backing
> file. I assume the backing file is consulted if the HTTP request for
> metadata fails. 

Yes, it is exactly that.  If the HTTP superclass fetchMetadata()
fails/throws (this is what executes the HTTP GET), then the backing
file is read in and returned as the new candidate metadata.  The code
that calls fetchMetadata() doesn't really know the difference, whether
was obtained via HTTP or from disk.


> Is the startup strategy different than the normal
> refresh strategy?

No, it's exactly the same.  The startup init process simply calls
refresh() and that's it.


The general order-of-operations is as follows, leaving out some details
irrelevant to your questions:

1) refresh() (includes initial startup) calls fetchMetdata()

2) fetchMetadata() attempts to load "new" metadata, via HTTP GET.  If
the HTTP request operation throws an exception (e.g. HTTP status !=
200, some other I/O error), then it loads the backing file from disk.
   - returns null byte[] if HTTP conditional GET indicates hasn't
changed from last time

3) if fetchMetadata successfully returned a null byte[], keep old
metadata and stop

4) byte[] is parsed to XMLObject

5) 'validUntil' check is applied. If invalid, keep the old metadata and
stop

6) apply metadata filter(s).  If not successful (depends on filters
configured, e.g. signature invalid, rejected for missing 'validUntil'),
keep old metadata and stop

7) swap in new metadata as effective and schedule next refresh


At all steps, when there is a fatal processing error/exception, then
the next refresh time is updated accordingly and the last successfully
loaded metadata is retained.


> Suppose the MetadataProvider contains the following filter:
>
> <MetadataFilter xsi:type="SignatureValidation">
>
> If signature verification fails, is the backing file consulted or does
> the process stop dead in its tracks?

Neither exactly.  Closer to the latter, but I wouldn't say it stops
dead, it does some housekeeping to schedule the next refresh attempt,
and retains the old metadata. Per the above, the filter is applied to
whatever was loaded (the new candidate metadata), which was either via
HTTP or from backing file.  It literally doesn't know the difference.


>
> Same question for the following filter:
>
> <MetadataFilter xsi:type="metadata:RequiredValidUntil">


Same answer.



You may be asking:  Why is metadata loaded from backing file on disk
implicitly re-validated and re-filtered?  The answer to the former is
obvious: it could have expired since it was saved.  The answer to the
latter is perhaps less so, but:  Aside from making the code consistent
and simpler (it's already complicated enough, believe me...), on a
software reload/restart the set of filters and/or the data they use to
operate may have changed: filters added/removed, trust material
changed, etc.  It re-applies the filters so that you know the backing
file loaded from disk is still accepted based on the current config. 
Some filters also modify the metadata (add entity attributes, remove
entities via whitelist/blacklist), so you have to re-process a new
metadata byte[] everytime you see one to get the data in the currently
expected form.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/dev/attachments/20160330/95919b09/attachment.html>