Metadata support: duplicate entityID detection; resolver lazy eval vs. preprocessing

[Cantor, Scott]

On 8/12/13 7:21 PM, "Brent Putman" <putmanb at georgetown.edu> wrote:
>
>Here I wanted to bring up the duplicates-within-a-single-provider case.
>To do detection there, the provider would obviously have to scan every
>EntityDescriptor in the collection.  In the case of the large
>federations' metadata, this might be very expensive.  I suppose it could
>do only the first time and cache the result, but it's still more
>expensive than the stop-on-first lookup algorithm that we have today.

I would have assumed it indexed up front.

>For example, today we do not traverse the
>whole document creating a comprehensive index of entityID ->
>EntityDescriptor.  We wait until an entityID is requested and then we
>find it and index it lazily.  This has advantages where a particular
>entity is say an IdP and is only ever going to communicate with SP's -
>there's no need to do all the work processing the IdP's in the metadata.

I suppose, but really a big deal? Plus which there are filters that
already walk the whole set anyway, aren't there?

>On the other hand, the first lookup for a given entityID takes longer
>than subsequent ones.  This would be eliminated if we pre-processed
>upfront.  The fetch and processing is in a background thread, so it
>doesn't affect perceived request time for callers, but would add to the
>load on the IdP, in some cases substantially where the metadata is huge.

I can't imagine this being that substantial, but I've never profiled the
SP.

>The v3 metadata redesign could have some other potential pre-processing
>work to do, such as pushing (pulling?) extensions from parent
>EntitiesDescriptors down to each child EntityDescriptor, and pushing the
>"annotation" or entity attribute of parent EntitiesDescriptors down to
>the descendent EntityDescriptors.  As far as I can determine right now,
>those operations can also be done lazily when a particular
>EntityDescriptor is resolved (with appropriate bits stored to only do it
>once, the first time), and avoid the cost of processing the whole
>document upfront.

Maybe, but I can see that causing some synchronization issues too.

>Mainly just throwing this out for comment, in case anyone sees any
>issues with that approach.  Doing lazy eval has the advantage of being
>what we do today; doing a bunch of document preprocessing on every new
>document fetch will also add more code and work, so trying to be
>pragmatic here.

I guess I'm saying I can't see why preprocessing should be a bad model,
but if we have code doing the right things now, then I don't think it has
to change.

-- Scott