<div dir="ltr"><div class="gmail_default" style="font-family:"times new roman",serif">Using the Prefork MPM since that is the default MPM with SLES and I really had not given it much thought. I'll check out the Event MPM and see if that would make sense for us to switch to.</div><div class="gmail_default" style="font-family:"times new roman",serif"><br></div><div class="gmail_default" style="font-family:"times new roman",serif">As for keep alive, it is on with the following settings:</div><div class="gmail_default"><ul><li><font face="times new roman, serif">MaxKeepAliveRequests 100<br></font></li><li><font face="times new roman, serif">KeepAliveTimeout 15<br></font></li></ul><font face="times new roman, serif">Anyways, the point of this was not to critique the settings but to say I found a way to reproduce the ELException. What exactly in the stress test is causing it I do not know, but it has consistently shown up a few times during the testing.</font></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><br><font face="times new roman, serif">Brian Moon</font><br><font face="times new roman, serif">Senior System Administrator</font><br><font face="times new roman, serif">Enterprise Systems</font><br><font face="times new roman, serif">Santa Clara University</font><br><font face="times new roman, serif">Office: 408.554.4830</font><br><a href="mailto:bmoon@scu.edu" style="font-family:'times new roman',serif" target="_blank">bmoon@scu.edu</a><br></div></div></div></div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Thu, May 18, 2017 at 5:20 PM, Cameron Kerr <span dir="ltr"><<a href="mailto:cameron.kerr@otago.ac.nz" target="_blank">cameron.kerr@otago.ac.nz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-NZ" link="blue" vlink="purple">
<div class="m_7372220404314353364WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">I would wonder why you use the Prefork MPM with Apache 2.4 instead of say the Event MPM, which is much more scalable. I would also
suggest that enabling Keepalives (with a reasonably short lifetime) would also help with scalability and cut down on port utilisation.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Cheers,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Cameron<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> users [mailto:<a href="mailto:users-bounces@shibboleth.net" target="_blank">users-bounces@<wbr>shibboleth.net</a>]
<b>On Behalf Of </b>Brian Moon<br>
<b>Sent:</b> Friday, 19 May 2017 11:54 AM<br>
<b>To:</b> Shib Users <<a href="mailto:users@shibboleth.net" target="_blank">users@shibboleth.net</a>><br>
<b>Subject:</b> Re: Rare ELException<u></u><u></u></span></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">Hello all,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">While stress testing one of our IdP servers today we were able to reproduce this error. Below is some information about our environment and the testing parameters:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">IdP Server:<u></u><u></u></p>
</div>
<div>
<ul type="disc">
<li class="MsoNormal">
SLES 12 SP2<u></u><u></u></li><li class="MsoNormal">
IdP Version 3.3.1<u></u><u></u></li><li class="MsoNormal">
Oracle JDK 1.8.0_112-b15<u></u><u></u></li><li class="MsoNormal">
Tomcat 8.0.36-20.7<u></u><u></u></li><li class="MsoNormal">
JAVA_OPTS: -Xms3g -Xmx3g -XX:MaxGCPauseMillis=400 -XX:+DisableExplicitGC -XX:+UseG1GC<u></u><u></u></li><li class="MsoNormal">
2 Cores<u></u><u></u></li><li class="MsoNormal">
6 GB RAM (VM memory reservation set to reserve all guest memory)<u></u><u></u></li><li class="MsoNormal">
Apache 2.4.23-21.1<u></u><u></u></li></ul>
<ul type="disc">
<ul type="circle">
<li class="MsoNormal">
Apache Prefork MPM<u></u><u></u></li><li class="MsoNormal">
ServerLimit 5120<u></u><u></u></li><li class="MsoNormal">
MaxClients 5120<u></u><u></u></li></ul>
</ul>
<p class="MsoNormal">Testing VM:<u></u><u></u></p>
</div>
<div>
<ul type="disc">
<li class="MsoNormal">
Windows 7 Enterprise (64-bit)<u></u><u></u></li><li class="MsoNormal">
4 Cores<u></u><u></u></li><li class="MsoNormal">
8 GB RAM<u></u><u></u></li><li class="MsoNormal">
Oracle JRE 1.8.0_131-b11<u></u><u></u></li><li class="MsoNormal">
Apache JMeter 3.2 r1790748<u></u><u></u></li></ul>
<ul type="disc">
<ul type="circle">
<li class="MsoNormal">
Test plan: <a href="https://wiki.shibboleth.net/confluence/download/attachments/17072886/shibboleth-unsolicited-sso-test.jmx?version=2&modificationDate=1435784363124&api=v2" target="_blank">https://wiki.shibboleth.<wbr>net/confluence/download/<wbr>attachments/17072886/<wbr>shibboleth-unsolicited-sso-<wbr>test.jmx?version=2&<wbr>modificationDate=<wbr>1435784363124&api=v2</a><u></u><u></u></li><li class="MsoNormal">
ThreadCount 1500<u></u><u></u></li><li class="MsoNormal">
Duration 300<u></u><u></u></li><li class="MsoNormal">
RampUpPeriod 10<u></u><u></u></li><li class="MsoNormal">
ProviderId <a href="https://sp.testshib.org/shibboleth-sp" target="_blank">https://sp.testshib.org/<wbr>shibboleth-sp</a><u></u><u></u></li></ul>
</ul>
<p class="MsoNormal">For the Apache config on the IdP server, the ServerLimit and MaxClients were set to something ridiculous so we could see what kind of load it could take. On the testing VM we were running out of ephemeral ports, so we added a registry
key to increase the max port number that could be used by a user: <a href="https://www.baselogic.com/blog/solved-java-net-bindexception-address-use-connect-issue-windows/" target="_blank">https://www.baselogic.<wbr>com/blog/solved-java-net-<wbr>bindexception-address-use-<wbr>connect-issue-windows/</a>
(this is not needed to reproduce the error, but cuts down on some noise). We did still have some socket errors on the testing VM, but overall it seemed to run a lot better.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Hope that helps in reproducing this and tracking down the cause.<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<u></u><u></u></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><br>
Brian Moon<br>
Senior System Administrator<br>
Enterprise Systems<br>
Santa Clara University<br>
Office: <a href="tel:(408)%20554-4830" value="+14085544830" target="_blank">408.554.4830</a><br>
<a href="mailto:bmoon@scu.edu" target="_blank">bmoon@scu.edu</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Mon, Apr 24, 2017 at 9:26 AM, Cantor, Scott <<a href="mailto:cantor.2@osu.edu" target="_blank">cantor.2@osu.edu</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">> Yes, I'm currently waiting for my session to timeout on the login form so I can<br>
> see what it does. I would expect it to throw a flow execution error though, so<br>
> I don't know if that's going to do it. I also see *very* few of these in my log,<br>
> like once a day or so, and that's with a decent volume, so that's puzzling.<br>
<br>
A basic "Wait too long" timeout doesn't have anything to do with this error, so that's as expected. So I have no idea how one would reproduce it.<br>
<span style="color:#888888"><br>
<span class="m_7372220404314353364hoenzb">-- Scott</span></span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><br>
<br>
--<br>
To unsubscribe from this list send an email to <a href="mailto:users-unsubscribe@shibboleth.net" target="_blank">
users-unsubscribe@shibboleth.<wbr>net</a><u></u><u></u></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div>
</div>
</div>
<br>--<br>
To unsubscribe from this list send an email to <a href="mailto:users-unsubscribe@shibboleth.net">users-unsubscribe@shibboleth.<wbr>net</a><br></blockquote></div><br></div>