Bad Message 431 in a single SP environment

Fri Sep 7 03:11:54 EDT 2018

Thanks, Nate

I haven't tried increasing the setting yet. It's a non-trivial exercise since I'll need to work out how to code it into the environment-specific section of our Shibboleth repository. But I'll do it if necessary.

I'll investigate some more based on your request line advice

Thanks again,

--
Kylie Lunghusen
Technical Tools Administrator, University Operations
Information Technology Services, RMIT University

________________________________
From: users <users-bounces at shibboleth.net> on behalf of Nate Klingenstein <ndk at sudonym.me>
Sent: Friday, 7 September 2018 2:46 PM
To: Shib Users
Subject: Re: Bad Message 431 in a single SP environment

Kylie,

That's a new one for me.  These are just some observations.

First, I think that's the right setting to be examining.  Have you tried simply increasing the maximum header size setting in Jetty in your test environment as a sanity check?

I would suppose that the reason it works on the first login but not subsequent logins is that the second cycle has an SSO session and other cookies set by the IdP, Jetty, or whatever else is involved in the first round-trip.

Second, according to the second answer at:

https://stackoverflow.com/questions/686217/maximum-on-http-header-values<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F686217%2Fmaximum-on-http-header-values&data=02%7C01%7C%7C7da9e02f9f4d4ceac8c708d6147d0a76%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636718924655545369&sdata=V1InXmOTIyK0m%2B0QD%2Bt5KDgSwIGKAsmsfeowrNuAwm8%3D&reserved=0>

the limit applies to the sum of the headers *and* the request line for most web servers, and as HTTP-Redirect bound SAML AuthnRequests can be of decent size, that 500 byte overhead is not that significant.  An example AuthnRequest from SAMLtest:

https://samltest.id/idp/profile/SAML2/Redirect/SSO?SAMLRequest=fZFfa4MwFMW%2FiuS9jdrWaqiCax9W6Fapbg97GVGvM6CJy4378%2B2ntYMORt8COed3OOdukLdNx%2BLe1PIE7z2gsb7aRiI7f4Sk15IpjgKZ5C0gMwVL44cDc%2Bc267QyqlANsWJE0EYouVUS%2BxZ0CvpDFPB0OoSkNqZDRulINEPAXJQ0rUWeqwZMPUdUdES6NDmmGbF2g0RIPtL%2B94qyo0N0JRq4GE9QCg2FoWl6JNZ%2BF5JX382XXrXIK8%2BrgFee7zi%2BGwSrvFytS75yBxliD3uJhksTEtd2%2FJkdzOx1Zi%2FZImAL94VYyaXhnZClkG%2B358gnEbL7LEtmU5ln0HguMghItBlrsHOwvpr5Npb%2Fbkui%2F9YY3xS7Db1iT0Edexxg%2B12iGlF8W3HTqM%2BtBm4gJA6h0WT5e%2FzoBw%3D%3D&RelayState=ss%3Amem%3A0002e12f35f8cedebf8403e8ac65e9c79522daf1d66940bc73088ba7c8296f65<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsamltest.id%2Fidp%2Fprofile%2FSAML2%2FRedirect%2FSSO%3FSAMLRequest%3DfZFfa4MwFMW%252FiuS9jdrWaqiCax9W6Fapbg97GVGvM6CJy4378%252B2ntYMORt8COed3OOdukLdNx%252BLe1PIE7z2gsb7aRiI7f4Sk15IpjgKZ5C0gMwVL44cDc%252Bc267QyqlANsWJE0EYouVUS%252BxZ0CvpDFPB0OoSkNqZDRulINEPAXJQ0rUWeqwZMPUdUdES6NDmmGbF2g0RIPtL%252B94qyo0N0JRq4GE9QCg2FoWl6JNZ%252BF5JX382XXrXIK8%252BrgFee7zi%252BGwSrvFytS75yBxliD3uJhksTEtd2%252FJkdzOx1Zi%252FZImAL94VYyaXhnZClkG%252B358gnEbL7LEtmU5ln0HguMghItBlrsHOwvpr5Npb%252Fbkui%252F9YY3xS7Db1iT0Edexxg%252B12iGlF8W3HTqM%252BtBm4gJA6h0WT5e%252FzoBw%253D%253D%26RelayState%3Dss%253Amem%253A0002e12f35f8cedebf8403e8ac65e9c79522daf1d66940bc73088ba7c8296f65&data=02%7C01%7C%7C7da9e02f9f4d4ceac8c708d6147d0a76%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636718924655555377&sdata=SNvsZuUN6b28z39DiYhF7SHr1jBbY8056Jbt%2FgUz2t0%3D&reserved=0>

is 604 bytes.  That would be just about right given the 150 byte discrepancy between test and prod and and 500 byte headroom.

That's where I'd start, but I haven't encountered this issue myself before, nor do I recall it coming up on the list before.

I hope this isn't too wrong,
Nate.

On Fri, Sep 7, 2018 at 2:32 AM, Kylie Lunghusen <kylie.lunghusen at rmit.edu.au<mailto:kylie.lunghusen at rmit.edu.au>> wrote:

Hi folks,

We've got a problem in which the Test and Production versions of a particular SP behave differently.

Background:
SP [ORG]-test.[APP].com.au uses our Test IdP
SP [ORG].[APP].com.au user our Production IdP
(Both IdPs defer to CAS/AD for authentication.)

Problem:
Production SP works fine.
On first login to the Test SP, it works. On subsequent logins, it gives the error:
"Bad Message 431
reason: Request Header Fields Too Large"
This behaviour is consistent, and applies across browsers and operating systems.

Investigations so far:
* I'm told the SPs are configured more or less identically.
* requestHeaderSize in jetty.xml is set to 8192 on all of our IdP environments.
* In second-access request headers captured via Dev Tools, the Test headers are usually slightly longer than the Prod ones, but still well below 8192 (eg. Test 7627, Prod 7458).

Only things I can think of are:
* requestHeaderSize is not the only setting (or is the wrong setting) to be checking?
* There are wrappers that make the thing bigger than it looks (like the way an email in transit is bigger than in the inbox)?
* Bytes != characters (I know some characters use two bytes, dunno if that includes any of these ones) so the Test headers really are over 8192?
(Apologies if stupid questions, am learning on the job with no mentor.)

I've failed to answer these questions via Googling, so it's time to ask the folks who know a lot more than I do.

Any ideas on what/where I should be checking?

Thanks,

K

--
Kylie Lunghusen
Technical Tools Administrator, University Operations
Information Technology Services, RMIT University

--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.shibboleth.net%2Fconfluence%2Fx%2FcoFAAg&data=02%7C01%7C%7C7da9e02f9f4d4ceac8c708d6147d0a76%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636718924655565389&sdata=PYMbviiXB3O3jMvBfgm2TF3eSUjuNq1Hr%2BVzpnF9co8%3D&reserved=0>
To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net<mailto:users-unsubscribe at shibboleth.net>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20180907/5546bc0c/attachment.html>