Handling Scope for AD as an IdP to linux shibboleth SPs

[Peter Schober]

Allan,

Managing trust between hundreds of SPs and "several" (roughly how many
are we talking here?) sounds to me more like the task of a federation
operator (think InCommon, but there are many smaller federations
around) than that of a typical SP admin. I might be explaining things
from that perspective below.

Btw, if those IDPs are / were part of InCommon you could consider
standing up a SAML proxy registered as an SP within InCommon, acting
as a single IDP towards your hundreds of SPs not registered with
InCommon. (I take it that registering hunderds of SPs within InCommon
would be prohibitively expensive.)
Then all of the work managing those IDPs' metadata has already be done
by the IDPs themselfs and/or InCommon.

Allan West via users <users at shibboleth.net> [2025-01-28 20:44 CET]:
> I manage several hundred linux / apache website SPs, and several of
> the remote IdPs we authenticate to are Microsoft Active Directory
> servers. If we give the raw AD IdP metadata to the SPs, then the SP
> won't authenticate users because the IdP fails to assert a Scope. We
> have one (1) partner who provides scoped metadata for their IdP and
> everyone else requires us to maintain a scoped copy of their
> metadata for them.

What specific attributes are we talking about here (where the scope
check fails): eduPersonScopedAffiliation? eduPersonPrincipalName? SAML
Subject-ID attributes? MS-ADFS systems have no idea about any of those
and MS-based systems might know/send attributes such as
userPrincipalName which appear to be scoped but are not formally.
(Why am I asking this? To me it makes a difference whether those IDPs
have been changed to assert defined-to-be-"scoped" attributes or
whether your SP was changed to perform scope checking on attributes
that are not actually defined to be scoped, such as UPN. Now whether
UPN and the likes *should* be scope-checked is a different question.)

As Scott said, the Shibboleth SP performs scope checking by default
(i.e., it can be configured differently) and it does so (again: by
default) only for attributes that are defined to be scoped in the
Shibboleth[1], eduPerson[2] or SAML Subject-ID[3] sense.

That scope checking prevents an SP from accepting "scoped" attribute
values (e.g. affiliations or identifiers) from one IDP that could
legitimately only have been asserted by another IDP.
  Importantly this will likely *not* include *all* *attributes* your
SPs care about, e.g. it does not (by default) include UPN nor will
email addresses be scope-checked.
  So your first question should be how much you care about that scope
checking, meaning, what attributes are your SPs relying on, which of
those are defined to be scoped (and consequently checked) and which
are not. How big is the risk of accepting some attributes without
scope checking (such as mail, name attributes, entitlements, group
information, etc.) and how big is the benefit of performing
scope checking only on the few attributes that are defined to be
scoped?
That could lead to deciding you don't care about scope checking in
which case you're done -- just remove that policy rule from your SPs.
  But obviously you can't benefit from scope checking without having
that additional layer (metadata extension on the IDPs) in place that
*authorizes* the scopes each IDP is allowed to assert scoped
attributes in.

Realizing that this amendment of IDP metadata is not at merely (or at
all) "doing the work of much bigger IDP organizations" but an
additional layer of control/authorization should make it clear that
this information shouldn't be purely self-asserted by those IDPs,
otherwise it would be mostly meaningless ("I authorize myself to
assert attributes in these scopes") or *completely* meaningless when
combined with automated metadata import from those IDPs, as you'd
never know (i.e., get to see) what scopes each new metadata import
would bring.
So this also depends on how you manage/import/verify the IDP metadata
your SPs consume.

The role of a federation operator (whether that's the job description
or not) ordinarily is that of a Trusted Third Party vetting both SPs
and IDPs. That also includes making sure each IDP has the right
metadata extension limiting what scopes they can assert
defined-to-be-scoped attributes in, in order to limit the SPs' risk of
falling victim to trivial (but hard to detect) impersonation, account
takover (including admin accounts, if those are federated, too) or
general data breaches.
So to me managing that extra piece of information is not unwanted
work, it's an essential piece of the trust framework I'm providing to
the IDPs and SPs (though mostly the SPs benefit from it) that chose to
participate.

> The MS page says that AD cannot add a scope to it's own IdP metadata
> and describes alternate ways to make your own.

And I wouldn't trust it if it were there, and neither should you if
you're "blindly"/automatically importing metadata from those IDPs.

> I've been doing it by hand in a text editor, and then leaving it to
> sit until the IdP owners reach out to say there's an update that we
> need to acknowledge.

Not sure that'll make you feel any better but that's one way to do it.
How bad that is depends on how bad it makes you feel vs. how much you
enjoy being able to wield the power of a proper text editor.

> Can someone please tell me whether there are:
>   1. automated tools for inserting a scope into AD IdP metadata?

Stepping up from making manual changes in a text editor would be
scripting that job. Suitable tools for that are either purely
XSLT-based (e.g. a parameterized XSL snippet plus xsltproc) or maybe
https://en.wikipedia.org/wiki/XMLStarlet or maybe doing it
programmatically (e.g. Python + lxml).

Provided you have a data structure (e.g. a simple YAML dictionary)
that enumerates the allowed scopes for each IDP entityID -- if you
don't then how would you know what scopes to add to what IDP metadata?
-- you'd simply iterate over all IDP entityIDs and add the allowed
scopes to each IDP's MetadataDescriptor (in a local file).
Then commit the result to git (or whatever). When regularly running
this script check for changes in the results using `git diff`. Commit
wanted changes (to "accept" and distribute metadata to the SPs,
manually or using gitops) or investigate as needed.

(That's also a recipe for managing metadata imports in general, if you
can't rely on an existing federation / federation operator having done
all that work for you.)

>   2. bypass options for the SP to ignore the AD IdPs' scope check?

Fully answered above, I hope. (You can do it but only after carefully
examining how much your SPs benefit from performing that check on the
selected attributes it applies to.)

>   3. shibboleth SP docs that give more details about working with AD IdPs?

Since this is a question of managing IDP metadata and to me more
generally one of managing "trust" at scale you won't find anything
about it in the SP software docs. (It's not an SP software issue.)

> I'm just checking before another land grant university asks mine to
> do all the work for them to make their lives easier.

As discussed above it's not for them to purely self-assert that (i.e.,
not without oversight).  They could do it but that wouldn't increase
the trust (depending on the mechanics of your metadata management).

> I just hate that I've never found a clever way to make AD IdPs work
> better with linux SPs.

This is not limited to MS-ADFS as SAML IDPs at all (the same goes for
almost all implementations) nor to "Linux" SPs: It applies to any SAML
SP that can be configured to perform scope checking, on any
OS/platform. (Which admittedly limits this to Shibboleth -- still the
only SP implementation to do that by default, SimpleSAMLphp and
SaToSa, I think.)

Btw, if you're serious about pushing more work to those IDPs a
complete re-tooling could present them with a self-service UI
(e.g. using https://jagger.heanet.ie/) that still enforces approval
steps for some changes you don't want to be purely self-asserted.
  Another approach could involve https://pyff.io/ to aggregate the
metadata from the IDPs, possibly transforming them with xslt or custom
Python code (pyff "pipelines") in the process, then sign the
aggregated IDP metadata before publishing it out to your SPs.
  But simply making metadata changes locally, committing those to a
git repo, checking remote IDP metadata for changes and git-rebasing
local changes on top of acceptable remote changes should go a long
way: You'd get the benefits of (semi-)automated remote metadata import
from the IDPs while managing/controlling scope extensions locally.

HTH,
-peter

[1] https://shibboleth.atlassian.net/wiki/spaces/CONCEPT/pages/928645231/NameIdentifiers
[2] https://wiki.refeds.org/display/STAN/eduPerson+%28202208%29+v4.4.0#eduPerson(202208)v4.4.0-Scope
[3] https://wiki.oasis-open.org/security/SAMLSubjectIDAttr