Duplicate xml:lang key invalidates metadata service

[Peter Schober]

I started to enter an issue into the bug tracker but it's unclear to
me where the issue lies and just how much the software is at fault
here:

Recently I fat-fingered a SAML 2.0 metadata change and introduced
duplicate xml:lang attributes in the MDUI extension (specifically
EntityDescriptor/SPSSODescriptor/Extensions/mdui:UIInfo/mdui:PrivacyStatementURL).
producing this:
<mdui:PrivacyStatementURL xml:lang="en">SOME_URL</mdui:PrivacyStatementURL>
<mdui:PrivacyStatementURL xml:lang="en">SOME_URL</mdui:PrivacyStatementURL>
instead of:
<mdui:PrivacyStatementURL xml:lang="de">SOME_URL</mdui:PrivacyStatementURL>
<mdui:PrivacyStatementURL xml:lang="en">SOME_URL</mdui:PrivacyStatementURL>
(So the xml:lang attribute values should have been different but were
not, though the element cdata values would have been identical for
both languages, the SP only has a single URL for multiple languages.)

Once this botch got published it broke an IDP that (1) had an
effective failFast=true set for the metadata service (reason for that
yet to be explained by the IDP operator) and (2) was also restarted at
the same time:

2021-02-26 12:23:27,611 - - ERROR
[org.opensaml.saml.metadata.resolver.impl.AbstractMetadataResolver:292]
- Metadata Resolver FileBackedHTTPMetadataResolver ACOnet: Metadata
provider failed to properly initialize, fail-fast=true, halting
2021-02-26 12:23:27,624 - - ERROR
[net.shibboleth.utilities.java.support.service.AbstractReloadableService:182]
- Service 'shibboleth.MetadataResolverService': Initial load failed

[...]

Exception during refresh
at
org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver.refresh(AbstractReloadingMetadataResolver.java:378)
Caused by:
java.lang.IllegalStateException: Duplicate key en (attempted merging
values SOME_URL and SOME_URL)
at
java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:133)
2021-02-26 12:23:27,624 - - ERROR
[net.shibboleth.utilities.java.support.service.AbstractReloadableService:185]
- Service 'shibboleth.MetadataResolverService': No further attempts will be made to reload

Now, clearly I was missing a test for just that case /and/ for an IDP
to break immediately after publishing that additionally required a
series of unfortunate events (non-default failfast setting in effect
somewhere, [possibly unrelated and/or ill-advised] restart attempt
around the same time).
*But* it seemed draconian for the IDP to fail on xml:lang on Extension
elements that are not operationally essential for any of the SAML
profiles.  Of course it /was/ set to failFast and the IDP /was/
restarted while the botch was still in the metadata.  And of course
the lower level code (java.util.stream.Collectors) has no idea about
what I consider essential and what kinds of errors I'd like it to
ignore, and it probably couldn't deal with duplicate keys in any other
way than to fail.

But I'll still ask here if there's anything that can be improved here.

If that one IDP had *not* been set to failFast (possibly by mistake)
and if no restart had additionally been attempted right after the
botch was published that same error would have gone unnoticed for the
remaining lifetime of our metadata and then "suddenly" all Shib IDPs
would have lost /all/ their SAML SPs -- including locally managed
ones, as it's the metadata service as such that fails, not just
individual metadata providers.

Best,
-peter