SP: de-duping resolver plugin

Tue Jun 19 14:47:24 BST 2012

Hi Scott,

thanks for taking the time to discuss this issue, very much appreciated.

Am 18.06.2012 19:31, schrieb Cantor, Scott:
> On 6/18/12 1:11 PM, "Martin Haase" <martin.haase at daasi.de> wrote:
>> Ok... So the options are deduplication "before" alias processing and
>> "after"?
> Aliases are built-in, there's no way around them.  One attribute could be
> named A and aliased to B and another could be named C and also aliased to
> B. If they overlap, you'd have no duplication of A or C, but would have a
> duplicate value in B. But the only way to de-duplicate B would be to
> destroy all the values for either A or C.
understood, although I only taught and never really used the alias feature.

> The closest to "before" and "after" is "during resolution" and "during
> export to variables". During resolution, you have the alias problem.
> During export, the aliases simply become independent variables to the
> application, so any modifications only affect the exported results, and
> not the underlying data.
>
> But before is much more efficient than after, because doing it in the
> export step means doing it on every web request. That doesn't bother me
> much, since I don't really think this feature is needed (or at least I
> haven't see the use case yet separate from the problem of multiple values
> in general).
>
> I can totally understand a desire to limit an "eppn" result to one value,
> since using aggregation can certainly give you duplicates. So saying
> "eppn" must have one value seems like a feature one might want. But that
> isn't about duplicates per se.
>
>> If possible I'd like to be application-independent. As the duplication
>> is "created" in the SP (by using multiple resolver plugins) and not in
>> the app, I think the duplication should be reverted there as well and
>> not in the app.
> The SP doesn't create duplicates only because of multiple plugins. It's
> inherent in the design, certainly in part because of aliases. It is always
> the assumption that you will have duplicates. The reasons are because it's
> virtually impossible to prevent them except by making destructive
> assumptions, or by doing it per-request in the export step.
>
>> Otherwise I'd need to instruct each app developer I will
>> give my consensus SP config to that they have to de-duplicate whatever
>> they get from the SP.
> Ok, but when would the application developer need to do this, but
> otherwise be fine handling multiple values?
I suspect with multiple emails? If the app is dumb it would enter all
available values into its database, including duplicates...

>> Fundamentally, I don't see how to implement (a). What is a duplicate?
>> What
>> if the attribute ID matches but the set of aliases is different? What if
>> the serialized form happens to match but the underlying attribute type
>> is
>> different? Etc.
>> I'd base it on the SAML Attribute Name and the serialized value(s).
> I can't do that. The SAML name is gone by the time I would see one.
I see.

>> ACK. How about putting the de-duplicator in the filtering stage?
> I looked there because that's really where the work should be, but it
> really doesn't work right now. The filtering stage doesn't get access to
> all of the attributes that have been previously resolved by different
> plugins. Changing that is more than I'm prepared to tackle in a minor
> update.
Maybe for another non-minor update?

>> What comes to my mind, could I make Attribute Extraction on a
>> per-relying party basis, would that be possible? The docs do not suggest
>> that.
> Depends what you mean. You can have a dedicated AttributeExtractor defined
> inside the SimpleAggregation resolver plugin. You can't currently have a
> different one that runs dynamically based on the source of the attributes.
> I don't know if that's enough for what you had in mind or not.
This could do the job, I'd have to check that.

>> And, I guess I have another workaround: Just NOT accept any attributes
> >from Campus IdPs but the ePPN or whatever links the identities to the
>> central IdP. Only use the central IdP's attributes. As you wrote in the
>> linked post from May 4, 2011, REMOTE_USER does de-duping, so that would
>> be fine...
> It does, but as I'm saying, the real point there is that it precludes
> multiple values altogether. That still seems to me to be a more relevant
> issue, and it's a pretty simply change (I think) to add an option to
> specify export variables that should be limited to one value.
Yeah, and preferably on a per-attribute base, because e.g. for mail or
givenName one might want to allow for multiple values, but for eppn not.

What about the following (also likely non-minor):
Define some rules in the filtering stage that say something like: "If
IdP X provided attribute A, take A from X and filter out all other
occurences of A. If IdP X didn't provide A, take A from IdP Y etc...".
This could have additional value besides just de-duplicating.

Regards,
Martin

>
> -- Scott
>
> --
> To unsubscribe from this list send an email to dev-unsubscribe at shibboleth.net

-- 
-----------------------------------------------------------------------
Dr. Martin Haase
DAASI International GmbH                   phone:     +49 7071 407109-6
Europaplatz 3                              Fax  :     +49 7071 407109-9
D-72072 Tübingen                           email: Martin.Haase at DAASI.de
Germany                                    Web  :   http://www.daasi.de

Directory Applications for Advanced Security and Information Management
-----------------------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2332 bytes
Desc: S/MIME Kryptografische Unterschrift
Url : http://shibboleth.net/pipermail/dev/attachments/20120619/31656ce0/attachment.bin 