MetadataProvider and update

[Peter Schober]

* Jan Vilhuber <JVilhuber at absolute.com> [2017-05-11 08:27]:
> I'm wondering about how best to implement something: In the UI of my
> app, for each customer, I want to be able to support them adding
> their IDP (upload metadata, manual entry, whatever). I need this new
> (or perhaps removed if they disable the IDP) information to be
> available on the Shibboleth-SP proxy. I realize I can have timed
> refreshes of the MetadataProvider (using url and refreshTimeout).

IIRC the SP automatically refreshes local metadata sources (i.e.,
MetadataProvivders of type "XML" or "Folder") if the local files
change.

So I take it you're currently pulling signed metadata via http(s) from
another server?

Try pushing the generated metadata to the SP as a local, preconfigured
file instead. As a side effect that also means you could do away with
signature validation MetadataFilters which will massively speed up
loading of changed metadata files, even very large ones.

> I don't want to set the refreshTimeout to something like 1 minute to
> make sure the SP sees the new IDP in a reasonable amount of time,
> since the other 29 days of the month will just be useless polling.

Just how much real damage would the most-often-useless polling cause?

> Is there a better mechanism?

Reading from a local file. Sadly the "Folder" approach currently does
not (re-)load files it hasn't seen on startup (i.e., it doesn't
monitor the folder for new files), otherwise you could just add,
modify or remove individual files to/in/from a configured folder.
(I guess patches implementing this will be welcome, should you decide
to want that.)

> I saw a mention of 'Dynamic Metadataprovider', which tries to fetch
> the metadata based on the entityId URL, but that requires me to
> validate the remotely fetched metadata (my current scheme publishes
> a federated metadata file, so I can validate it with a single
> certificate), and also requires all IDP's to publish the metadata at
> that URL (I think.. haven't read all the details of the dynamic
> approach), which I expect isn't widely done (is it?).

There are two different mechanisms here, both implemented in the SP's
"Dynamic" MetadataProvider:

One is where you fetch SAML Metadata from the URL values that is an
entity's entityID (the "well-known location). That's not common,
mostly because self-hosting self-asserted metadata is meaningless (not
trustworthy), and self-hosting third party-signed metadata is
difficult (cf. letsencrypt). So that's not what you want.

The other dynamic scheme is asking a single Metadata "oracle" (a
server of yours, whose signature on resulting SAML Metadata you trust,
same as with asking for an aggregate, but now only asking for a single
entity's EntityDescriptor), based on a query for the entityID.
Cf. https://github.com/iay/md-query
For SP configuration this is done simply by adding the 'Subst' child
element to the "Dynamic" MetadataProvider, cf.
https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPMetadataProvider#NativeSPMetadataProvider-ChildElements.1
(And no, that functionality is not at all obvious from the docs. ;))

Personally I'd go with reading all IDPs from a single local file,
provisioned there via whatever methods you chose to implement.
If that file becomes too big look at md-query. You'll move the logic
and implementation of signing to the md-query server, then, but you
have great flexibility there.
(E.g. it could be doing the generation and signing in real-time, on
request from your SP. You could also pre-sign individual
EntityDescriptors and store them as individual files on disk on the
md-query server, as serving up static" files from a web server can be
extremely fast. Then add, modify, remove only the changed IDPs on the
md-query server. Though that assumes you can also sign dynamically,
for the adds and modifications. That's usually where some form of HSM
comes into play, but that depends on your requirements and how you
satisfy those at the moment.)

HTH,
-peter