shibd Memory Usage

[Cantor, Scott]

On 9/26/14, 11:20 AM, "Andy Bennett" <andyjpb at knodium.com> wrote:
>
>We tend to find that it likes to use a lot of RAM. I always assumed it
>was just a function of the number of transactions. However, I've just
>looked on one of our machines that doesn't actually do any Shibboleth
>stuff but has the same configuration installed as our other machines.
>It's been sitting there for 600 days or so and the logs show that all
>it's been doing is downloading the metadata feed: it hasn't processed
>any transactions. That shib was using some 700MiB of RAM.

That's not all that shocking given that Linux, unlike Windows, does not
free heap. This is known as "broken", but it dates from the standard Unix
philosophy that threads are evil and process churn isn't.

That said, you can certainly check and make sure you're not running Apache
prefork to rule that out as a source of RAM spikes.

>Is that the amount that would be expected (given the size of the
>ukfederation metadata of a few thousand entries) or is something amiss?

I don't really know what it uses on a routine basis, but all it takes is a
spike, because once it's allocated, it ain't coming back. It's virtual of
course, and empty heap will just end up on disk, but it's still going to
be allocated.

There were a few small leaks fixed in 2.5.x, IIRC.

-- Scott