Overloaded shibd questions

[Cantor, Scott]

On 5/19/21, 3:29 PM, "users on behalf of Nate Klingenstein" <users-bounces at shibboleth.net on behalf of ndk at signet.id> wrote:

>    So, we're in a situation where a large number of sessions and modules are tethered to a single shibd server
> with a TCPListener on a secured network for clustering due to earlier expedient deployment decisions and an
> unexpectedly rapidly growing client base.

It's almost like when I tell people that doesn't scale I might be right.

>  Sometimes after login and ACS processing and redirect, Shibboleth attributes will be populated.  Sometimes
> they won't be.  The application servers are not under any appreciable load and have been extensively laterally
> scaled.

Unless you're dealing with, maybe, passive rules, that's not possible, the requests just don't go further if the session lookup fails, and they'd bounce back to the IdP (that would be a loop, perhaps).

>    The CPU is getting pegged on the shibd server, but I'm looking for confirmation that that's the root cause of
> attributes occasionally not being set, and if the failure mode is 500 errors rather than attributes not getting set
> and the request being passed through, then I'm going to have to revisit assumptions.

I didn't think even a passive rule would pass along no attributes, but it's not implausible there might be such a code path since that approach implies it's ok not to have a session.

>    The application will itself invoke a loop if the variables are unpopulated because it considers it an
> unauthenticated request.  It's checking for presence of a specific attribute to determine authentication status,
> which I've verified is sent by the IdP and mapped by Shibboleth.  Logs look like:

i.e. yes, this is passive. So yes, probably there's some way in that might be possible.

>    Yes.  I was thinking this could be used as a stopgap to reduce the load on shibd.  But if it's really just a
> recovery hack that is only useful for that purpose alone, that kills the idea.

Well, it's a hack, but it works. It's certainly more practical than a database for clustering. But you can't expect to constantly switch nodes and not see it kill performance to some extent. The SP cannot run without stickiness, period.

-- Scott