IdP OutOfMemoryError on refresh of UKFederation Metadata

[Peter Schober]

* Robert Hardy (r.hardy) <r.hardy at yorksj.ac.uk> [2021-11-13 20:13]:
> The issue does not seem to be a lack of memory - shibd_idpw shows
> max memory pool set to 2048MB.

I haven't tried consuming large, signed aggregates with 2 GB for the
JVM in recent times.  My own IDP consuming something very much like
the UKfederation aggregate runs fine with 3GB assigned to Java
(leaving 1GB for the rest of the minimal OS). Rock solid.

> We are still on IdP version 3.3.0.

You'd needed to check the issue tracker for signs of the IDP having
had memory leaks in unsupported versions of that vintage but I
wouldn't excpect that to be the case. (At least I never managed to
trigger any when I was on that version.)

FWIW, I've recently helped someone debug why their IDP wasn't running
stably (needed periodic restarts or was running out of memory; I can't
recall what the specific pathological behaviour was) and it turned out
that they had dozens (nearing hundreds) of individual MetadataProvider
elements configured (one for each SP) and therefore reloaded the
complete Metadata subsystem -- including the one MetadataProvider that
loaded all of eduGAIN/UKfederaion -- every time they added (or
removed) an SP.
So I'd expect some anti-pattern of (mis-)using the software to be a
likely source of the issue. Though that's not something you can easily
check for in your own deployment unless you already knew it was an
anti-pattern (in which case you'd probably have avoided configuring
the software that way), I suppose.

-peter