SP: de-duping resolver plugin

[Cantor, Scott]

On 6/18/12 1:11 PM, "Martin Haase" <martin.haase at daasi.de> wrote:
>Ok... So the options are deduplication "before" alias processing and
>"after"?

Aliases are built-in, there's no way around them.  One attribute could be
named A and aliased to B and another could be named C and also aliased to
B. If they overlap, you'd have no duplication of A or C, but would have a
duplicate value in B. But the only way to de-duplicate B would be to
destroy all the values for either A or C.

The closest to "before" and "after" is "during resolution" and "during
export to variables". During resolution, you have the alias problem.
During export, the aliases simply become independent variables to the
application, so any modifications only affect the exported results, and
not the underlying data.

But before is much more efficient than after, because doing it in the
export step means doing it on every web request. That doesn't bother me
much, since I don't really think this feature is needed (or at least I
haven't see the use case yet separate from the problem of multiple values
in general).

I can totally understand a desire to limit an "eppn" result to one value,
since using aggregation can certainly give you duplicates. So saying
"eppn" must have one value seems like a feature one might want. But that
isn't about duplicates per se.

>If possible I'd like to be application-independent. As the duplication
>is "created" in the SP (by using multiple resolver plugins) and not in
>the app, I think the duplication should be reverted there as well and
>not in the app.

The SP doesn't create duplicates only because of multiple plugins. It's
inherent in the design, certainly in part because of aliases. It is always
the assumption that you will have duplicates. The reasons are because it's
virtually impossible to prevent them except by making destructive
assumptions, or by doing it per-request in the export step.

> Otherwise I'd need to instruct each app developer I will
>give my consensus SP config to that they have to de-duplicate whatever
>they get from the SP.

Ok, but when would the application developer need to do this, but
otherwise be fine handling multiple values?

> The cases you draw up hold of course: only if the
>app supports multiple values it could have a chance in de-duplicating.
>But the point is that architecturally it should not be handled by the app.

The issue is whether it needs to be handled at all, not where.

>>Fundamentally, I don't see how to implement (a). What is a duplicate?
>>What
>> if the attribute ID matches but the set of aliases is different? What if
>> the serialized form happens to match but the underlying attribute type
>>is
>> different? Etc.

>I'd base it on the SAML Attribute Name and the serialized value(s).

I can't do that. The SAML name is gone by the time I would see one.

>ACK. How about putting the de-duplicator in the filtering stage?

I looked there because that's really where the work should be, but it
really doesn't work right now. The filtering stage doesn't get access to
all of the attributes that have been previously resolved by different
plugins. Changing that is more than I'm prepared to tackle in a minor
update.

>What comes to my mind, could I make Attribute Extraction on a
>per-relying party basis, would that be possible? The docs do not suggest
>that.

Depends what you mean. You can have a dedicated AttributeExtractor defined
inside the SimpleAggregation resolver plugin. You can't currently have a
different one that runs dynamically based on the source of the attributes.
I don't know if that's enough for what you had in mind or not.

>And, I guess I have another workaround: Just NOT accept any attributes
>from Campus IdPs but the ePPN or whatever links the identities to the
>central IdP. Only use the central IdP's attributes. As you wrote in the
>linked post from May 4, 2011, REMOTE_USER does de-duping, so that would
>be fine...

It does, but as I'm saying, the real point there is that it precludes
multiple values altogether. That still seems to me to be a more relevant
issue, and it's a pretty simply change (I think) to add an option to
specify export variables that should be limited to one value.

-- Scott