testing errors

[Tom Scavo]

tl;dr Since the combined deployment footprint for the IdP + SP is
substantial, TestShib makes it easier for deployers to get started (as
evidenced by its popularity). That said, for the most part TestShib is
a missed opportunity to reinforce the methods of trusted metadata
exchange.

On Tue, Jan 16, 2018 at 11:21 AM, Cantor, Scott <cantor.2 at osu.edu> wrote:
>
> People don't need a metadata service.

That’s debatable. (I’ll come back to this.)

> They're trying to avoid running an IdP or SP, that's testshib's purpose. I don't happen to like or agree with that, but that's what people are (mis)using it for, to avoid the responsibility that comes with operating a SSO system.

I was going to throw up my hands and declare total ignorance about why
deployers flock to TestShib but your observation makes sense to me so
let’s start there.

> You run both halves to test and understand how it works, and people just don't want to.

If there were a Java SP component, I think you could argue that point
successfully, but without that, I don’t think that’s a reasonable
expectation. The combined deployment footprint for the IdP + SP is
substantial.

We entered the long tail of SAML software deployment a long time ago.
Basically what I hear you saying is that small IT shops (which is all
that remains on the IdP side) should rely on a hosted solution (which
may or may not be based on the Shibboleth software). I guess that’s
okay but it would be better if the project just came out and said that
if in fact that’s the goal.

> I have always advocated that we should eliminate the metadata component and run testshib as an open system that doesn't require any metadata, and if it ever became something the project operated, that's what would happen.

That seems counterproductive. The initial deployment exercise (which
is basically an exercise in social engineering) is a critical exercise
in effective metadata exchange. It sets the stage for everything else
that follows. If the deployer cuts corners the first time, and it
works, then that’s what they’ll do the next time, and the time after
that, etc.

In terms of that crucial, initial deployment exercise, IdP deployers
and SP deployers have different needs of course. Let’s start from the
PoV of the IdP deployer (which is simplest).

There are numerous SPs in eduGAIN metadata just waiting for fresh, new
IdPs to come along. As you know, the problem is to find a clean way to
achieve automated (and trusted) metadata exchange. Unless the IdP
organization joins a federation (which is a high bar in some cases),
automated metadata exchange will remain a challenge.

Suppose TestShib had an SP component that accepted test IdP metadata
in some way. (The current method works fine as long as the IdP
metadata is not aggregated on the server.) Assuming the TestShib SP
metadata appears in the eduGAIN aggregate, the IdP deployer simply
consumes federation metadata (which is what we want them to do anyway)
to close the loop. Win-win, all the way around.

The TestShib SP might expose endpoints to test various levels of
attribute release, thereby reinforcing the prevailing party line with
respect to attribute release. It could also expose endpoints that
tested IsPassive, ForceAuthn, and RequestedAuthnContext.

Earlier you claimed that deployers don’t need another metadata
service. Maybe not another one but a better one. Suppose, for example,
that TestShib served test IdP metadata via the MDQ protocol (no
aggregates). Using that TestShib feature, the IdP deployer could
conduct further tests with any suitably configured SP in eduGAIN
metadata. To prevent the well-known InQueue syndrome, that metadata
would have a well-defined validity window.

All of this is my own opinion of course: If you downgrade TestShib, or
scare people away by heaping guilt upon them, you’ll end up shooting
yourself in the foot.

Tom