IDP 5 - NoSuchConversationException when clicking URL link from email and _saml_idp cookie set.

Tue Nov 21 23:43:54 UTC 2023

Hey Scott,

There is no CDN in play, and I can reproduce it with the Web Developer
Tools running and the disable caching flag enabled.  In addition, I
enforce Cache-Control headers to ensure none of these requests are
cached by the browser or any intermediate reverse proxies in the
picture.

I will explain in more detail the specific scenario for you.  In our
application, a user will be sent emails when a report is ready for
them to view, so they will get an email with the link to a  reports
servlet in our application which displays their reports.  For example,
the link in the email would be something like
https://www.foo.com/secure/myapp/myreports.  /secure is a path that is
protected by the SP.  Not sure if it's important to mention, but for
this specific path, we set the forceAuthn flag on the SAML request
because the information in the reports contains sensitive information
and we want to make sure they are forced to log in regardless if they
have an existing IDP session. If the user doesn't have a saml_idp
cookie set before clicking the link in the email, then the SP
intercepts the request and the SP redirects them to the DS, they
select their IDP, they are returned back to the SP with a URL similar
to...

https://www.foo.com/Shibboleth.sso/Login?SAMLDS=1&target=ss:mem:d61e985338c4c2f95403adsfasdfasa9d481f9ffa5asd3cbfd5b9c10ded&forceAuthn=true&entityID=https://www.foo.com/idp

And the SP sets the following cookie...

_opensaml_req_ss%3Amem%3A
d61e985338c4c2f95403adsfasdfasa9d481f9ffa5asd3cbfd5b9c10ded=_329cad87c4asdfasdf07a00d6eb0;
path=/; secure; HttpOnly; SameSite=None

Next, the user is redirected to the IDP
https://www.foo.com/idp/profile/SAML2/Redirect/SSO, with the following
URL params...

SAMLRequest
RelayState
SigAlg
Signature

The IDP responds to this request with a redirect to GET
/idp/profile/SAML2/Redirect/SSO?execution=e1s1, for example:

location:/idp/profile/SAML2/Redirect/SSO?execution=e1s1
set-cookie: __Host-JSESSIONID=8D0A05ASsB6DA2CA8C6.gfdsgfdtomcatrouteid;
Path=/; Secure; HttpOnly; SameSite=Strict

This results in CSRF POST to /idp/profile/SAML2/Redirect/SSO?execution=e1s1

csrf_token"_c0256f9f45asdfasdf48624f78eaasdf900552"
shib_idp_ls_exception.shib_idp_session_ss""
shib_idp_ls_success.shib_idp_session_ss"true"
shib_idp_ls_value.shib_idp_session_ss""
shib_idp_ls_exception.shib_idp_persistent_ss""
shib_idp_ls_success.shib_idp_persistent_ss"true"
shib_idp_ls_value.shib_idp_persistent_ss""
shib_idp_ls_supported"true"
_eventId_proceed""

The response to the CSRF POST is then redirected to
/idp/profile/SAML2/Redirect/SSO?execution=e1s2

At this point, they are at the login screen and can enter their
credentials.  I won't bore you with the rest since I'm sure you know
how it all works, but they end up redirected back to the SP with their
assertion, and the SP sets the _opensaml_req_ss, _shibsession, and
_saml_idp cookies.

set-cookie: _saml_idp=aHR0cHM6Ly9jcmlzZGV2LmRvasdfsdfasdfasdfcmUvdHhkb3RpZHAvc2hpYmJvbGV0aA%3D%3D;
path=/; secure; expires=Wed, 20 Nov 2024 20:26:23 GMT; path=/; secure;
HttpOnly

Then the SP runs the attribute checker, etc.. and redirects the user
to the original URL.

Then the user logs out (not SLO), just SP log out.

Here is where we see the new issue we haven't seen in the past.
Remember that forcedAuthn is enabled on our SP for the requested URL,
and now, unlike the initial request, the user has a _saml_idp cookie
that doesn't expire for a year; we do this because we got too many
complaints about always being asked to select their IDP when their
initial choice is not going to change.

The user gets a new email with a link to
https://www.foo.com/secure/myapp/myreports, but the difference is they
now have the _saml_idp cookie.

Here's what happens...

The SP sees the _saml_idp cookie, then immediately redirects the user
to the IDP https://www.foo.com/idp/profile/SAML2/Redirect/SSO with the
following URL parameters,
SAMLRequest
RelayState
SigAlg
Signature

and sets the _opensaml_req_ss cookie.

set-cookie: _opensaml_req_ss%3Amem%3A3dasdfasdfe689e9f6b093asdfasdfasdfa61ed9f99e23f1b513f0da7d8e998=_4092b18asdfasdfas9a8asdf4de94700;
path=/; secure; HttpOnly; SameSite=None

The request to https://www.foo.com/idp/profile/SAML2/Redirect/SSO has
the following cookies in the request
_opensaml_req_ss%3Amem%3A3dasdfasdfe689e9f6b093asdfasdfasdfa61ed9f99e23f1b513f0da7d8e998=_4092b18asdfasdfas9a8asdf4de94700
_saml_idp=aHR0cHM6Ly9jcmlzZGV2LmRvasdfsdfasdfasdfcmUvdHhkb3RpZHAvc2hpYmJvbGV0aA%3D%3D

The response sets the __Host-JSESSIONID cookie and then redirects the
user using a location header...

set-cookie: __Host-JSESSIONID=8367EAA6B7Aa6Easdfasdf4sdfasdfas.tomcatrouteid;
Path=/; Secure; HttpOnly; SameSite=Strict
location: /idp/profile/SAML2/Redirect/SSO?execution=e1s1

The problem is caused because the browser isn't sending the
__Host-JSESSIONID cookie in the request to
/idp/profile/SAML2/Redirect/SSO?execution=e1s1.

If I use Charles Proxy to add the __Host-JSESSIONID cookie on the
request, then things work, so I now know that not having the
__Host-JSESSIONID cookie on the request is what causes the IDP error.

When I compare the requests, I notice that the value of
"sec-fetch-site" changes from "cross-site" to "same-origin" after the
SP redirects to the DS. So, by the time the request reaches the point
where the "__Host-JSESSIONID" is set, "sec-fetch-site" has changed to
"same-origin". However, on the requests where the "__saml_idp" is set
and the request is redirected to the IDP without going through the DS,
the "sec-fetch-site" remains "cross-site". I am not sure if this is
related, but according to the Set-Cookie documentation, there is no
explanation as to why the browser fails to send the
"__Host-JSESSIONID" cookie. The request to
https://www.foo.com/idp/profile/SAML2/Redirect/SSO sets the
"__Host-JSESSIONID" cookie, and the browser displays it in the list of
cookies it has for the host. The redirect to
"/idp/profile/SAML2/Redirect/SSO?execution=e1s1" should contain the
"__Host-JSESSIONID" cookie, as the scheme, domain, and path remain the
same, but it is not sent. I would assume that it is a browser bug, but
all the latest versions of Safari, Firefox, and Chrome behave in the
same way. There is no reference in the Set-Cookie specifications that
indicates that "sec-fetch-site" should affect the cookie being sent.
The cookie is set and can be seen in the browser storage, but it is
not sent with the request.

Thanks,

Dan

On Tue, Nov 21, 2023 at 9:41 AM Cantor, Scott <cantor.2 at osu.edu> wrote:
>
> Your message presents two conflicting claims: It both says that the discovery cookie (which doesn't inherently obviously have anything to do with the IdP) causes the error and then later the message says it prevents the error.
>
> My guess is you're seeing CDN caching. I have seen cases where an SP's redirect to the IdP gets cached by CDNs and they start presenting stale redirects to clients. Usually it causes a replayed AuthnRequest, but in principle it could just as easily end up playing back a URL from later in the conversation.
>
> By definition, there is no circumstance in which an initial request to the IdP with the execution key would ever work (as there is no conversation yet).
>
> The first request has to be with the SAML parameters, not a webflow key.
>
> The discovery cookie if it's not set could bypass that by forcing the client to hit the DS first, busting the cache.
>
> -- Scott
>
>