verifying security used by a new SP

[Cantor, Scott]

On 4/18/19, 3:33 PM, "users on behalf of Steven Carmody" <users-bounces at shibboleth.net on behalf of steven_carmody at brown.edu> wrote:

> What's the current recommended practice, when dealing with an SP that 
> you don't control or manage ? We'll be releasing a set of attributes to 
> this SP, and have the usual concerns in that situation.

I don't worry myself all that much about attributes because it's usually directory data or equally non-sensitive information I'm sending, and the terms of engagement are contracts that are way above my pay grade. Our default rules cover the non-contractual cases.

If I were to start judging any company's trustworthiness to care for our data based on their understanding of SAML, or pretty much anything in the vein of basic IT competence, I wouldn't be releasing much data.

But if you want a long answer...

These kinds of things are why I never, ever rely on any remote metadata sources directly from SPs, and why I'm careful with how I deal with their keys. If an SP botches a key rollover, I start working on avoiding their keys. I worry about the quality of their code and whether I have to pen test it for vulnerabilities like the truncation attacks and that kind of thing.

Signing requests (apart from logout, which most of them don't support anyway) makes it hard to avoid relying on their keys, and that has implications if they start changing keys annually and screwing that up. That's why it's a bad practice in general, it's a needless risk that offers no benefit. It's also a sign that they may be improperly failing to check things like an AuthnContext class they might be requesting (rare, but it happens) because they think signing the request somehow prevents manipulations of the IdP.

I focus mostly on maintaining a clean configuration, avoiding exceptions, pushing back on all bugs and mistakes I see even though it costs me a lot of time and hassle doing that (not time in terms of knowing what to do but time waiting for their responses and then arguing about it). I will usually test any "special" requirement I'm handed because they're usually not true. For example, if you tell me I have to sign assertions, I will not assume that and I'll try it the normal way first, and then if I'm right I fix their metadata and move on. Same for NameID Formats, I have not one actual use of the "unspecified" format, so when I say you don't need it, I have proof that's mostly true.

People think I'm a cynic or negative or rude or whatever, but my practical experience is that most vendors aren't competent, and they prove me largely right on a weekly basis, so I guess what I'm saying is, I don't go by some sort of baseline because none of them meet it. I know they're terrible, and I document each case so I know what I'm dealing with, I save all my emails, and I make a note of any red flags so that I'm not blindsided later. But all of this is a matter of years of experience spotting the danger signs and knowing what they mean.

An example tip: any SP not using Shibboleth I will generally check their certificate if they have a key and see if it's commercial or short lived. That tells me they a) don't know what they're doing and b) are going to screw me by changing their key, which they don't know how to do properly. That's a simple check that generally provides useful information up front. If I know for a fact they're going to botch the system, I'll turn off encryption up front and avoid the problem later.

I assume "wrong" and let the few exceptions give me the occasional pleasant surprise, and that generally works out better than trying to start from a position of trust.

-- Scott