shibd massive memory growth and CPU usage

[Cantor, Scott]

On 7/10/14, 8:30 PM, "Stockley, Jonathan" <jonathan.stockley at emc.com>
wrote:

>All goes well for a few days then they run the tests again and I see
>about 100+ shibd threads and VIRT and RES both go up to 3.5G and total
>CPU goes to about 700%, ie 7 of the 8 CPUs are pegged.

And are you using Apache 1.3 or prefork mode? If so, either stop, or
change the thread stack size.

The other issue is with load testing if the same NameID is sent over and
over. The SP will not handle that well, and it's not a real world
condition. You can work around that while testing by turning
maintainReverseIndex off in the SessionCache, and the bottleneck will be
avoided.

But seeing 100+ threads is a strong indicator of prefork.

> 
>It seems to me that sessions (that will never be used again) are building
>up far faster than they expire and get cleaned out.

Well, load testing will do that, but they will go away. I doubt that's the
issue.

>I¹m thinking that maybe I need to set Session timeout to something small,
>around 10 and reduce the SessionCache inprocTimeout to 10,
>cleanupInterval to 60 and then set the memory cleanupInterval to 60 as
>well.
> 
>Am I in the right ball park? Any ideas?

Those values are in seconds, so no, that would not be useful.

>I¹m also being asked ³How will we handle 1000s of simultaneous logins?²

Simultaneous sessions and logins are very different things, but answer #1
is to not use prefork.

-- Scott