IIS Module Intermittently Failing to Query Daemon over a TCPListener for Session Information

[Nate Klingenstein]

All,

This is a continuation of the saga of the passively protected application that initiates another login attempt after a successful SAML transaction that doesn't present any Shibboleth variables to the app, causing severe looping.  We think we know what's happening now.

We have upgraded to use the module instead of the filter, but remain on a 3.1.x release of the SP and a shared shibd.  Upgrades are being performed, but it's a large deployment and we hope this issue could be examined in parallel to that process for faster resolution.

As Scott noted, there is a code path by which the module could pass a user through to the application without retrieving a session successfully, but that it would spit out a lot of warnings and errors in the logs that would be impossible to miss.  We confirmed both that the session variables are not being set and that there are no such errors showing up in the logs, which was really confounding.

We added stickiness to the application servers, which reduced the load, but did not resolve the looping issue.

For deeper examination, we used Wireshark to examine traffic between the module and shibd, which is still operating on a separate server.  We found that on instances where the looping occurs(very sporadic, and not always for the same user, and eventually resolving itself with a successful login when the session variables are populated), the module does not even attempt to retrieve session information from shibd.  No packets are sent to shibd for a find::storage::sessioncache command in these instances, and obviously shibd doesn't spontaneously send anything to the module.  The module then passes the user to the application with no variables, and loop.

Messages are sent and received on successful logins.  This would explain why we were both seeing no errors in the logs and no variables being set in isolated incidents in this setup.

The looping is highly specific to individual session initiation attempts and can occur even while other users are logging in just fine on their first attempt.  It's seriously detrimental to performance and is right now being addressed by selectively and temporarily blocking users who are stuck in a loop, which is obviously not a fun solution.

Has anyone else encountered anything remotely like this before, or is there a hypothesis as to how and why this could be happening?

Thanks in advance,
Nate.

--------
Signet, Inc.
The Art of Access ®

https://www.signet.id