preloading high-value entity metadata

[Tom Scavo]

I added Example 4 to the ChainingMetadataProvider topic. [1] The
example shows how to preload "high-value SP metadata" using one or
more providers of type FileBackedHTTPMetadataProvider. Given the
robustness of a FileBackedHTTPMetadataProvider, this is a natural
thing to do. I know of at least one federation that explicitly
recommends such a strategy.

Taking this to its logical conclusion, we can predict: using
FileBackedHTTPMetadataProvider for "high-value SPs" will not scale in
the same way that FilesystemMetadataProvider does not scale for local
metadata. In going from Example #1 to Example #2 in the wiki, multiple
providers of type FilesystemMetadataProvider are replaced by a single
LocalDynamicMetadataProvider (which is a Good Thing). We need
something similar as we go from Example #3 to Example #4 (if we go
that way at all).

Consider the following hypothetical situation. Suppose there were a
provider called DirectoryBackedHTTPMetadataProvider:

- The provider has an attribute called backingDir, which is a
directory containing high-value entity metadata.

- As a reloading provider, the metadata in the backingDir is kept
fresh by periodically iterating over the backing files. In each case,
the metadata request URL is constructed by applying a transform to the
entityID (just like a dynamic provider).

- The reloading provider supports the same child elements as the
DynamicHTTPMetadataProvider.

So the provider is both a dynamic provider and a reloading provider.
Actually a new provider type is not strictly necessary, just add the
backingDir attribute to the DynamicHTTPMetadataProvider type. Whenever
a new entityID is encountered, spool a copy of entity metadata into
the backingDir.

I intentionally skipped all the details to keep this brief. Does it
make sense to extend DynamicHTTPMetadataProvider in this way?

Tom

[1] https://wiki.shibboleth.net/confluence/x/NgInAQ