Bad Message 431 in a single SP environment

Tue Sep 11 20:26:45 EDT 2018

Thanks, Nate and Terry

Sounds like it's best if I just nudge up the config to 16384 and be done with it,

--
Kylie Lunghusen
Technical Tools Administrator, University Operations
Information Technology Services, RMIT University

________________________________
From: users <users-bounces at shibboleth.net> on behalf of Terry Smith <t.smith at aaf.edu.au>
Sent: Tuesday, 11 September 2018 1:39 PM
To: Shib Users
Subject: Re: Bad Message 431 in a single SP environment

Hi Kylie,

We had a similar issue at another site. We increased setting on both Apache and Jetty as we have Apache proxying through to Jetty.

On Apache,

    LimitRequestFieldSize 16384
    LimitRequestLine 16384
    ProxyIOBufferSize 16384

On Jetty,

      <Set name="requestHeaderSize"><Property name="jetty.httpConfig.requestHeaderSize" deprecated="jetty.request.header.size" default="16384" /></Set>

After these changes there were no more incidents of Error 431 occurring from either Apache or Jetty.

Thanks,
Terry.

On Sat, 8 Sep 2018 at 03:24 Nate Klingenstein <ndk at sudonym.me<mailto:ndk at sudonym.me>> wrote:
Kylie,

If it's hard to change that setting, there may be a trivial way to test my hunch, although it won't work in production.  Just chop off most of the SAMLRequest parameter on the delivery of the second round trip(e.g. delete most of the query string) and you might find yourself at the IdP with an unparseable message, but evidence that Jetty counts the request line against the header limit size like most web servers.

It's not a fix, but it would point you towards one, and give you reason to go through the exercise of either shrinking some cookies or increasing the limit

Best wishes,
Nate.

On Fri, Sep 7, 2018 at 7:11 AM, Kylie Lunghusen <kylie.lunghusen at rmit.edu.au<mailto:kylie.lunghusen at rmit.edu.au>> wrote:

Thanks, Nate

I haven't tried increasing the setting yet. It's a non-trivial exercise since I'll need to work out how to code it into the environment-specific section of our Shibboleth repository. But I'll do it if necessary.

I'll investigate some more based on your request line advice

Thanks again,

--
Kylie Lunghusen
Technical Tools Administrator, University Operations
Information Technology Services, RMIT University

________________________________
From: users <users-bounces at shibboleth.net<mailto:users-bounces at shibboleth.net>> on behalf of Nate Klingenstein <ndk at sudonym.me<mailto:ndk at sudonym.me>>
Sent: Friday, 7 September 2018 2:46 PM
To: Shib Users
Subject: Re: Bad Message 431 in a single SP environment

Kylie,

That's a new one for me.  These are just some observations.

First, I think that's the right setting to be examining.  Have you tried simply increasing the maximum header size setting in Jetty in your test environment as a sanity check?

I would suppose that the reason it works on the first login but not subsequent logins is that the second cycle has an SSO session and other cookies set by the IdP, Jetty, or whatever else is involved in the first round-trip.

Second, according to the second answer at:

https://stackoverflow.com/questions/686217/maximum-on-http-header-values<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F686217%2Fmaximum-on-http-header-values&data=02%7C01%7C%7C7da9e02f9f4d4ceac8c708d6147d0a76%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636718924655545369&sdata=V1InXmOTIyK0m%2B0QD%2Bt5KDgSwIGKAsmsfeowrNuAwm8%3D&reserved=0>

the limit applies to the sum of the headers *and* the request line for most web servers, and as HTTP-Redirect bound SAML AuthnRequests can be of decent size, that 500 byte overhead is not that significant.  An example AuthnRequest from SAMLtest:

https://samltest.id/idp/profile/SAML2/Redirect/SSO?SAMLRequest=fZFfa4MwFMW%2FiuS9jdrWaqiCax9W6Fapbg97GVGvM6CJy4378%2B2ntYMORt8COed3OOdukLdNx%2BLe1PIE7z2gsb7aRiI7f4Sk15IpjgKZ5C0gMwVL44cDc%2Bc267QyqlANsWJE0EYouVUS%2BxZ0CvpDFPB0OoSkNqZDRulINEPAXJQ0rUWeqwZMPUdUdES6NDmmGbF2g0RIPtL%2B94qyo0N0JRq4GE9QCg2FoWl6JNZ%2BF5JX382XXrXIK8%2BrgFee7zi%2BGwSrvFytS75yBxliD3uJhksTEtd2%2FJkdzOx1Zi%2FZImAL94VYyaXhnZClkG%2B358gnEbL7LEtmU5ln0HguMghItBlrsHOwvpr5Npb%2Fbkui%2F9YY3xS7Db1iT0Edexxg%2B12iGlF8W3HTqM%2BtBm4gJA6h0WT5e%2FzoBw%3D%3D&RelayState=ss%3Amem%3A0002e12f35f8cedebf8403e8ac65e9c79522daf1d66940bc73088ba7c8296f65<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsamltest.id%2Fidp%2Fprofile%2FSAML2%2FRedirect%2FSSO%3FSAMLRequest%3DfZFfa4MwFMW%252FiuS9jdrWaqiCax9W6Fapbg97GVGvM6CJy4378%252B2ntYMORt8COed3OOdukLdNx%252BLe1PIE7z2gsb7aRiI7f4Sk15IpjgKZ5C0gMwVL44cDc%252Bc267QyqlANsWJE0EYouVUS%252BxZ0CvpDFPB0OoSkNqZDRulINEPAXJQ0rUWeqwZMPUdUdES6NDmmGbF2g0RIPtL%252B94qyo0N0JRq4GE9QCg2FoWl6JNZ%252BF5JX382XXrXIK8%252BrgFee7zi%252BGwSrvFytS75yBxliD3uJhksTEtd2%252FJkdzOx1Zi%252FZImAL94VYyaXhnZClkG%252B358gnEbL7LEtmU5ln0HguMghItBlrsHOwvpr5Npb%252Fbkui%252F9YY3xS7Db1iT0Edexxg%252B12iGlF8W3HTqM%252BtBm4gJA6h0WT5e%252FzoBw%253D%253D%26RelayState%3Dss%253Amem%253A0002e12f35f8cedebf8403e8ac65e9c79522daf1d66940bc73088ba7c8296f65&data=02%7C01%7C%7C7da9e02f9f4d4ceac8c708d6147d0a76%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636718924655555377&sdata=SNvsZuUN6b28z39DiYhF7SHr1jBbY8056Jbt%2FgUz2t0%3D&reserved=0>

is 604 bytes.  That would be just about right given the 150 byte discrepancy between test and prod and and 500 byte headroom.

That's where I'd start, but I haven't encountered this issue myself before, nor do I recall it coming up on the list before.

I hope this isn't too wrong,
Nate.

On Fri, Sep 7, 2018 at 2:32 AM, Kylie Lunghusen <kylie.lunghusen at rmit.edu.au<mailto:kylie.lunghusen at rmit.edu.au>> wrote:

Hi folks,

We've got a problem in which the Test and Production versions of a particular SP behave differently.

Background:
SP [ORG]-test.[APP].com.au uses our Test IdP
SP [ORG].[APP].com.au user our Production IdP
(Both IdPs defer to CAS/AD for authentication.)

Problem:
Production SP works fine.
On first login to the Test SP, it works. On subsequent logins, it gives the error:
"Bad Message 431
reason: Request Header Fields Too Large"
This behaviour is consistent, and applies across browsers and operating systems.

Investigations so far:
* I'm told the SPs are configured more or less identically.
* requestHeaderSize in jetty.xml is set to 8192 on all of our IdP environments.
* In second-access request headers captured via Dev Tools, the Test headers are usually slightly longer than the Prod ones, but still well below 8192 (eg. Test 7627, Prod 7458).

Only things I can think of are:
* requestHeaderSize is not the only setting (or is the wrong setting) to be checking?
* There are wrappers that make the thing bigger than it looks (like the way an email in transit is bigger than in the inbox)?
* Bytes != characters (I know some characters use two bytes, dunno if that includes any of these ones) so the Test headers really are over 8192?
(Apologies if stupid questions, am learning on the job with no mentor.)

I've failed to answer these questions via Googling, so it's time to ask the folks who know a lot more than I do.

Any ideas on what/where I should be checking?

Thanks,

K

--
Kylie Lunghusen
Technical Tools Administrator, University Operations
Information Technology Services, RMIT University

--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.shibboleth.net%2Fconfluence%2Fx%2FcoFAAg&data=02%7C01%7C%7C7da9e02f9f4d4ceac8c708d6147d0a76%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636718924655565389&sdata=PYMbviiXB3O3jMvBfgm2TF3eSUjuNq1Hr%2BVzpnF9co8%3D&reserved=0>
To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net<mailto:users-unsubscribe at shibboleth.net>

--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.shibboleth.net%2Fconfluence%2Fx%2FcoFAAg&data=02%7C01%7C%7C601e4c52f6504a76863d08d617985b45%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636722340517936834&sdata=3dKwfr9n3IwuaqCg%2Fdm2gWda8KsdGSXjo0liHG5CnQo%3D&reserved=0>
To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net<mailto:users-unsubscribe at shibboleth.net>

--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.shibboleth.net%2Fconfluence%2Fx%2FcoFAAg&data=02%7C01%7C%7C601e4c52f6504a76863d08d617985b45%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636722340517936834&sdata=3dKwfr9n3IwuaqCg%2Fdm2gWda8KsdGSXjo0liHG5CnQo%3D&reserved=0>
To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net<mailto:users-unsubscribe at shibboleth.net>
--

Terry Smith | Technical Engagement and Support Manager |  Australian Access Federation Inc

Mob: 0414 692 424<tel:0402%20918%20566> | Email: t.smith at aaf.edu.au<mailto:elleina.filippi at aaf.edu.au> | Address: Lvl 21, 179 Turbot St, Brisbane QLD 4000<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmaps.google.com%2F%3Fq%3DLvl%2B21%2C%2B179%2BTurbot%2BSt%2C%2BBrisbane%2BQLD%2B4000%26entry%3Dgmail%26source%3Dg&data=02%7C01%7C%7C601e4c52f6504a76863d08d617985b45%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636722340517936834&sdata=p5cQUF2VhT1JVbp0YT2r5g7SHcPE9b44u5eUDafmJ38%3D&reserved=0> |

Web: www.aaf.edu.au<https://apac01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.aaf.edu.au%2F&data=02%7C01%7C%7C601e4c52f6504a76863d08d617985b45%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636722340517936834&sdata=bBM%2FrcByCMiDOFSbz6SaEBin3DkedHIAYpOVH7NJN5U%3D&reserved=0> | Support: support.aaf.edu.au<https://apac01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsupport.aaf.edu.au%2F&data=02%7C01%7C%7C601e4c52f6504a76863d08d617985b45%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636722340517936834&sdata=PXQ8W8bg8tjbvXXlf%2Fawnb%2FFx2BoxZILilIEp%2FBuKJ8%3D&reserved=0> | Twitter: twitter.com/ausaccessfed<https://apac01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftwitter.com%2Fausaccessfed&data=02%7C01%7C%7C601e4c52f6504a76863d08d617985b45%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636722340517936834&sdata=zcXgsG67zsColc7L8stjUeWc4XQ9sPKAtyzDxv3t1CA%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20180912/dd84c493/attachment.html>