HTTPResource and multiple Metadata Provider files

[Cantor, Scott]

> ​so an example of an Metadatata attribute filter we want to use is:

Yes, I assumed so. But that is *inside* a MetadataResolver. So you have a resolver, and you tell it where to get metadata from, e.g. InCommon. So it loads *all* that metadata, and then it runs the filter(s) you define inside the MetadataResolver.

If you have multiple MetadataResolvers, I don't care where, one file, three, whatever, each of those resolvers is loading some set of metadata, and then applying filters to that metadata. There's no relationship *between* MetadataResolvers.

At the very top level, you have a chain of resolvers, from across all the different configuration resources loaded into that service, usually just one (metadata-providers.xml), and that lets it search all the resolvers one at a time.

I didn't *think* we specify an order across resources, but I'm looking at the code and now I see some kind of sortKey thing. So, bottom line, I don't know. I simply don't think it's a good idea. Duplicates are not a goal of this system, you don't want to load the same metadata twice.

> ​Even if the first filter in each file has something like:
>         <MetadataFilter xsi:type="Predicate" direction="include"
> removeEmptyEntitiesDescriptors="true" trim="true">
>           <Entity>https://sp-test.idm.ucsc.edu/shibboleth</Entity>
>         </MetadataFilter>

You keep referring to the filters as being "in files", but that's not how they work unless we implemented something I'm unaware of. They're in *resolvers* in files. The resolvers are what are loading the metadata, and then applying filters *within* that resolver's metadata.

So I'm having trouble connecting what you're describing with the way it works. I don't see anything in the parsers that supports loading a filter from a separate Spring resource. Like we did in the attribute resolver with those springResourceRef hooks. I don't think those exist for metadata filters. Or I could be wrong.

Within a given resolver, yes, if one filter throws out a ton of entities, the remaining filters don't have to go plowing through everything that was thrown out. But I'm not sure if that's what you're doing, because you were referencing separate files.

> I know it has to load the file, but then I thought it throws away all the entities
> that are not in the ​<Entity> lists

Within a single resolver, yes.

> ​Okay but the idea is that someone outside of our local university wants to be
> able to make decisions on our behalf for "some" entities about what's being
> requested/released, However it's still up to the institution to decide what
> the requested population means.​

Right, but the problem is that unless we did something I'm not familiar with, that "outside" agent needs to supply you with the *entire* MetadataResolver it wants you to use. There's no way to remotely reference a set of *filters* alone. Not that I know of.

> ​However I'm hearing you say about the order. It seems to be working the
> way I described now but you are saying that's just by chance.​

If you have two files containing MetadataResolvers, I don't think there is any guarantee as to the order they'll be checked, but now I don't know. If there is, ok. It certainly wouldn't be by accident, you would have to be using some kind of sort value to control it.

> ​Okay, so an alternate thought we have one metadata-providers.xml file and
> we have a process that "sed"'s a list of Metadata filters that add attributes
> inline, where the list of filters comes from somewhere else.​

Obviously with enough textual magic, you can manage anything, but I'm not aware of any supported features for piecemealing together a complete MetadataResolver with filters with bits of it coming from different places and resources. Not without writing code to do all the wiring up at runtime. Declaratively, I don't think our schema can do that, not that I can see.

Probably it *could* be extended to. We have other places in the code where we injected references to arbitrary beans, and there's no inherent reason that can't be done with metadata filters, we just didn't have a use for it.

It would really help to see what in the heck you're trying to load to answer anything specifically.

-- Scott