the relationship between metadata size and heap size

[Tom Scavo]

On Fri, Sep 11, 2015 at 9:31 PM, Cantor, Scott <cantor.2 at osu.edu> wrote:
>
> You should make sure, though, that you have adequate heap allocated to the JVM.

I've taken Scott's advice out of context and started a new thread. The
advice was given to an IdP deployer who reported a failed metadata
refresh process. [1] The process thread died quietly, which eventually
resulted in expired metadata and user failures.

I don't think this is an isolated incident. The same thing has
happened a handful of times in the InCommon Federation over the last
couple of months. No explanation was found at the time. At least now
we have a working theory.

The size of the InCommon metadata aggregate is now 16MB. Perhaps the
file has crossed a threshold such that some typical JVM deployments
are now exhibiting issues. If there's a relationship between aggregate
size and heap size, I surely would like to know what it is.

What makes this worse is that the InCommon metadata aggregate will
double in size overnight (literally) once we start importing eduGAIN
metadata. If I knew what to tell deployers to prepare for this event,
I would. Is there some JVM configuration that is known to work with
large aggregates of size 32MB and beyond?

In retrospect, this is not unlike the experience of simpleSAMLphp
deployers. As aggregate have grown in size, SSP deployers have found
that PHP needs more memory (surprise). The larger the aggregate, the
more memory that's needed.

I guess none of this will be too surprising...unless you happen to be
one of the affected deployers.

Every time I mention per-entity metadata and dynamic metadata query to
someone, they quote the familiar adage that Shibboleth can handle much
larger aggregates than we have today. It's time to stop fooling
ourselves. This is Yet Another Reason to start taking per-entity
metadata seriously, especially for IdPs.

Tom

[1] http://marc.info/?l=shibboleth-users&m=144196122923278&w=2