Bad Message 431 in a single SP environment

[Terry Smith]

Hi Kylie,

We had a similar issue at another site. We increased setting on both Apache
and Jetty as we have Apache proxying through to Jetty.

On Apache,

    LimitRequestFieldSize 16384
    LimitRequestLine 16384
    ProxyIOBufferSize 16384

On Jetty,

      <Set name="requestHeaderSize"><Property
name="jetty.httpConfig.requestHeaderSize"
deprecated="jetty.request.header.size" default="16384" /></Set>

After these changes there were no more incidents of Error 431 occurring
from either Apache or Jetty.

Thanks,
Terry.

On Sat, 8 Sep 2018 at 03:24 Nate Klingenstein <ndk at sudonym.me> wrote:

> Kylie,
>
> If it's hard to change that setting, there may be a trivial way to test my
> hunch, although it won't work in production.  Just chop off most of the
> SAMLRequest parameter on the delivery of the second round trip(e.g. delete
> most of the query string) and you might find yourself at the IdP with an
> unparseable message, but evidence that Jetty counts the request line
> against the header limit size like most web servers.
>
> It's not a fix, but it would point you towards one, and give you reason to
> go through the exercise of either shrinking some cookies or increasing the
> limit
>
> Best wishes,
> Nate.
>
> On Fri, Sep 7, 2018 at 7:11 AM, Kylie Lunghusen <
> kylie.lunghusen at rmit.edu.au> wrote:
>
>> Thanks, Nate
>>
>>
>> I haven't tried increasing the setting yet. It's a non-trivial exercise
>> since I'll need to work out how to code it into the environment-specific
>> section of our Shibboleth repository. But I'll do it if necessary.
>>
>>
>> I'll investigate some more based on your request line advice
>>
>>
>> Thanks again,
>>
>>
>>
>> --
>> Kylie Lunghusen
>> Technical Tools Administrator, University Operations
>> Information Technology Services, RMIT University
>>
>>
>>
>>
>> ------------------------------
>> *From:* users <users-bounces at shibboleth.net> on behalf of Nate
>> Klingenstein <ndk at sudonym.me>
>> *Sent:* Friday, 7 September 2018 2:46 PM
>> *To:* Shib Users
>> *Subject:* Re: Bad Message 431 in a single SP environment
>>
>> Kylie,
>>
>> That's a new one for me.  These are just some observations.
>>
>> First, I think that's the right setting to be examining.  Have you tried
>> simply increasing the maximum header size setting in Jetty in your test
>> environment as a sanity check?
>>
>> I would suppose that the reason it works on the first login but not
>> subsequent logins is that the second cycle has an SSO session and other
>> cookies set by the IdP, Jetty, or whatever else is involved in the first
>> round-trip.
>>
>> Second, according to the second answer at:
>>
>> https://stackoverflow.com/questions/686217/maximum-on-http-header-values
>> <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F686217%2Fmaximum-on-http-header-values&data=02%7C01%7C%7C7da9e02f9f4d4ceac8c708d6147d0a76%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636718924655545369&sdata=V1InXmOTIyK0m%2B0QD%2Bt5KDgSwIGKAsmsfeowrNuAwm8%3D&reserved=0>
>>
>> the limit applies to the sum of the headers *and* the request line for
>> most web servers, and as HTTP-Redirect bound SAML AuthnRequests can be of
>> decent size, that 500 byte overhead is not that significant.  An example
>> AuthnRequest from SAMLtest:
>>
>>
>> https://samltest.id/idp/profile/SAML2/Redirect/SSO?SAMLRequest=fZFfa4MwFMW%2FiuS9jdrWaqiCax9W6Fapbg97GVGvM6CJy4378%2B2ntYMORt8COed3OOdukLdNx%2BLe1PIE7z2gsb7aRiI7f4Sk15IpjgKZ5C0gMwVL44cDc%2Bc267QyqlANsWJE0EYouVUS%2BxZ0CvpDFPB0OoSkNqZDRulINEPAXJQ0rUWeqwZMPUdUdES6NDmmGbF2g0RIPtL%2B94qyo0N0JRq4GE9QCg2FoWl6JNZ%2BF5JX382XXrXIK8%2BrgFee7zi%2BGwSrvFytS75yBxliD3uJhksTEtd2%2FJkdzOx1Zi%2FZImAL94VYyaXhnZClkG%2B358gnEbL7LEtmU5ln0HguMghItBlrsHOwvpr5Npb%2Fbkui%2F9YY3xS7Db1iT0Edexxg%2B12iGlF8W3HTqM%2BtBm4gJA6h0WT5e%2FzoBw%3D%3D&RelayState=ss%3Amem%3A0002e12f35f8cedebf8403e8ac65e9c79522daf1d66940bc73088ba7c8296f65
>> <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsamltest.id%2Fidp%2Fprofile%2FSAML2%2FRedirect%2FSSO%3FSAMLRequest%3DfZFfa4MwFMW%252FiuS9jdrWaqiCax9W6Fapbg97GVGvM6CJy4378%252B2ntYMORt8COed3OOdukLdNx%252BLe1PIE7z2gsb7aRiI7f4Sk15IpjgKZ5C0gMwVL44cDc%252Bc267QyqlANsWJE0EYouVUS%252BxZ0CvpDFPB0OoSkNqZDRulINEPAXJQ0rUWeqwZMPUdUdES6NDmmGbF2g0RIPtL%252B94qyo0N0JRq4GE9QCg2FoWl6JNZ%252BF5JX382XXrXIK8%252BrgFee7zi%252BGwSrvFytS75yBxliD3uJhksTEtd2%252FJkdzOx1Zi%252FZImAL94VYyaXhnZClkG%252B358gnEbL7LEtmU5ln0HguMghItBlrsHOwvpr5Npb%252Fbkui%252F9YY3xS7Db1iT0Edexxg%252B12iGlF8W3HTqM%252BtBm4gJA6h0WT5e%252FzoBw%253D%253D%26RelayState%3Dss%253Amem%253A0002e12f35f8cedebf8403e8ac65e9c79522daf1d66940bc73088ba7c8296f65&data=02%7C01%7C%7C7da9e02f9f4d4ceac8c708d6147d0a76%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636718924655555377&sdata=SNvsZuUN6b28z39DiYhF7SHr1jBbY8056Jbt%2FgUz2t0%3D&reserved=0>
>>
>> is 604 bytes.  That would be just about right given the 150 byte
>> discrepancy between test and prod and and 500 byte headroom.
>>
>> That's where I'd start, but I haven't encountered this issue myself
>> before, nor do I recall it coming up on the list before.
>>
>> I hope this isn't too wrong,
>> Nate.
>>
>>
>>
>> On Fri, Sep 7, 2018 at 2:32 AM, Kylie Lunghusen <
>> kylie.lunghusen at rmit.edu.au> wrote:
>>
>> Hi folks,
>>
>> We've got a problem in which the Test and Production versions of a
>> particular SP behave differently.
>>
>> Background:
>> SP [ORG]-test.[APP].com.au uses our Test IdP
>> SP [ORG].[APP].com.au user our Production IdP
>> (Both IdPs defer to CAS/AD for authentication.)
>>
>> Problem:
>> Production SP works fine.
>> On first login to the Test SP, it works. On subsequent logins, it gives
>> the error:
>> "Bad Message 431
>> reason: Request Header Fields Too Large"
>> This behaviour is consistent, and applies across browsers and operating
>> systems.
>>
>> Investigations so far:
>> * I'm told the SPs are configured more or less identically.
>> * requestHeaderSize in jetty.xml is set to 8192 on all of our IdP
>> environments.
>> * In second-access request headers captured via Dev Tools, the Test
>> headers are usually slightly longer than the Prod ones, but still well
>> below 8192 (eg. Test 7627, Prod 7458).
>>
>> Only things I can think of are:
>> * requestHeaderSize is not the only setting (or is the wrong setting) to
>> be checking?
>> * There are wrappers that make the thing bigger than it looks (like the
>> way an email in transit is bigger than in the inbox)?
>> * Bytes != characters (I know some characters use two bytes, dunno if
>> that includes any of these ones) so the Test headers really are over 8192?
>> (Apologies if stupid questions, am learning on the job with no mentor.)
>>
>> I've failed to answer these questions via Googling, so it's time to ask
>> the folks who know a lot more than I do.
>>
>> Any ideas on what/where I should be checking?
>>
>> Thanks,
>>
>> K
>>
>>
>> --
>> Kylie Lunghusen
>> Technical Tools Administrator, University Operations
>> Information Technology Services, RMIT University
>>
>>
>>
>> --
>> For Consortium Member technical support, see
>> https://wiki.shibboleth.net/confluence/x/coFAAg
>> <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.shibboleth.net%2Fconfluence%2Fx%2FcoFAAg&data=02%7C01%7C%7C7da9e02f9f4d4ceac8c708d6147d0a76%7Cd1323671cdbe4417b4d4bdb24b51316b%7C0%7C0%7C636718924655565389&sdata=PYMbviiXB3O3jMvBfgm2TF3eSUjuNq1Hr%2BVzpnF9co8%3D&reserved=0>
>> To unsubscribe from this list send an email to
>> users-unsubscribe at shibboleth.net
>>
>>
>>
>> --
>> For Consortium Member technical support, see
>> https://wiki.shibboleth.net/confluence/x/coFAAg
>> To unsubscribe from this list send an email to
>> users-unsubscribe at shibboleth.net
>>
>
> --
> For Consortium Member technical support, see
> https://wiki.shibboleth.net/confluence/x/coFAAg
> To unsubscribe from this list send an email to
> users-unsubscribe at shibboleth.net

-- 

*Terry Smith* | Technical Engagement and Support Manager |  Australian
Access Federation Inc

*Mob: *0414 692 424 <0402%20918%20566> | *Email**: t.smith*@aaf.edu.au
<elleina.filippi at aaf.edu.au> | *Address*: Lvl 21, 179 Turbot St, Brisbane
QLD 4000
<https://maps.google.com/?q=Lvl+21,+179+Turbot+St,+Brisbane+QLD+4000&entry=gmail&source=g>
 |

*Web:* www.aaf.edu.au |* Support:* support.aaf.edu.au | *Twitter:*
twitter.com/ausaccessfed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20180911/beacd4ef/attachment.html>