shibd Memory Usage

[Cantor, Scott]

On 9/26/14, 12:11 PM, "Andy Bennett" <andyjpb at knodium.com> wrote:
>
>Thanks for the info. My restarted shibd is already at 388MiB Virtual and
>148MiB Resident.

I just restarted it on shibboleth.net, which loads a lot of metadata.
After a single load to start up, it's at 312M. So, basically, that's just
how it is until the federations switch to on-demand metadata. DOM loads
are expensive, and unlike on Windows, the memory doesn't come back. That
same memory footprint on Windows will shrink down to under 100M because
the DOMs dropping frees up a ton of heap.

Under most conditions, it shouldn't grow enormously because each DOM load
reuses most of the earlier heap blocks, but there's always some growth
over time.

>I've not seen any other long running processes eat into swap like this
>but maybe it's a function of the frequently reallocated metadata
>structures?

Yes. I forced Xerces to tune that down a bit, but it's still bad. And I
don't even know what Xerces you're using. I think the worst numbers are
far back in history, but I'd be leery of anything before 3.0.

-- Scott