<div dir="ltr">On further investigation, I think I had misdiagnosed the problem. We were loading the metadata with a typical FileBackedHTTPMetadataProvider. E.g.:<div><br></div><div><font face="monospace"> <MetadataProvider xsi:type="FileBackedHTTPMetadataProvider"<br> id="FOO"<br> metadataURL="<a href="https://foo.example.com/saml/metadata/sp">https://foo.example.com/saml/metadata/sp</a>"<br> backingFile="%{idp.home}/metadata/foo-metadata.xml" /></font><br><br></div><div>However, while poking around, I noticed an existing backingFile that was empty with size 0. I'm unsure how we came to have this empty backingFile.</div><div><br></div><div>The logged error was</div><div><br></div><div><font face="monospace">ERROR [org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver:376] - Metadata Resolver FileBackedHTTPMetadataResolver FOO: Error occurred while attempting to refresh metadata from '<a href="https://foo.example.com/saml/metadata/sp">https://foo.example.com/saml/metadata/sp</a>'<br>net.shibboleth.utilities.java.support.resolver.ResolverException: Unable to unmarshall metadata<br> at org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver.unmarshallMetadata(AbstractReloadingMetadataResolver.java:462)<br>Caused by: org.opensaml.core.xml.io.UnmarshallingException: net.shibboleth.utilities.java.support.xml.XMLParserException: Unable to parse inputstream, it contained invalid XML<br> at org.opensaml.saml.metadata.resolver.impl.AbstractMetadataResolver.unmarshallMetadata(AbstractMetadataResolver.java:356)<br>Caused by: net.shibboleth.utilities.java.support.xml.XMLParserException: Unable to parse inputstream, it contained invalid XML<br> at net.shibboleth.utilities.java.support.xml.BasicParserPool.parse(BasicParserPool.java:253)<br>Caused by: org.xml.sax.SAXParseException: Premature end of file.<br> at java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:204)</font><br></div><div><br></div><div>I'd assumed the "Unable to parse inputstream, it contained invalid XML" was occuring while trying to load the metadata from the metadataURL, but I think it was actually occuring while trying to parse the existing empty backingFile(?), which is consistent with "Premature end of file" (and it certainly "contained" invalid XML). So I'm guessing the IdP was trying to check the backingFile contents (to unmarshall it?) before refreshing it from the metadataURL. After deleting the empty backingFile, the IdP was able to successfully load the metadata without error from the metadataURL, and it apparently didn't balk at the "
" entities in the downloaded metadata.</div><div><br></div><div>Mea culpa.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 23, 2022 at 11:57 PM Ian Young <<a href="mailto:ian@iay.org.uk">ian@iay.org.uk</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space"><br><div><blockquote type="cite"><div>On 2022-03-24, at 01:59, Jason Pyeron <<a href="mailto:jpyeron@pdinc.us" target="_blank">jpyeron@pdinc.us</a>> wrote:</div><br><div><span style="color:rgb(31,73,125);font-family:Calibri,sans-serif;font-size:14.666666984558105px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">RFC 7468 allows for whitespace, 
 is ASCII 13, a white space char. Why not here? The RFC says, in General Considerations:</span></div></blockquote></div><div><br></div>Things are probably a bit more complicated than they appear. When we see an encoded CR character like this, either as 
 or as it's usually because something in the chain got deeply confused about how the end of a line should be represented.<div><br></div><div>On Windows, as you're probably aware, text file lines are CR (12) followed by LF (10). On most other systems these days, it's just LF. If something gets moved from one kind of system to another, the CR can end up isolated from the logical end of line and treated as independent white space.</div><div><br></div><div>This sounds harmless (it's just white space, in some sense) but in fact can cause issues with things like signatures (on one system, there's just a line terminator, on another it's white space then a line terminator).</div><div><br></div><div>For example, see <a href="https://shibboleth.atlassian.net/browse/SSPCPP-684" target="_blank">https://shibboleth.atlassian.net/browse/SSPCPP-684</a></div><div><br></div><div>I didn't think the IdP was similarly impacted but I guess I'm not astonished. As a result of the problems with the SP, several federations now look for this and exclude entity-encoded CR characters to avoid issues for their members, so we don't see this in the wild any more.</div><div><br></div><div>As to this:</div><div><br></div><div><blockquote type="cite">"Since we include the "
" characters to show a break in the metadata information, we will not be able to remove these characters from the XML file"</blockquote><div><br></div>Your supplier is just obviously confused, sorry. The "
" does nothing of the kind. A line break does, and this may once have been part of a line break, but it isn't any more.<br><div><br></div><div>As you say this just appears in the certificate data, I'd have to say my suspicion is that they got bitten by one of the changes that was made to Base64 handling in Java a while back, in which things like certificate and signature data ends up internally with CR LF sequences inside the Java strings; when written out on a non-Windows system, the CRs end up encoded. In my opinion, this is nonsensical and a result of an over-reading of the Base64 specification but obviously others disagree.</div><div><br></div><div>If your supplier thinks this is somehow mandated by the XML standard "to show a break in the metadata information", I wonder why they didn't notice that the rest of the XML wasn't done the same way.</div><div><br></div><div>Having said which, if that's their response I doubt that you'll be able to have a real conversation about this. You're probably going to have to remove the entities by hand.</div><div><br></div><div><blockquote type="cite">The IdP fails to parse this. Is this actually valid SAML for the metadata?</blockquote><br></div><div>It's complicated. It might be worth putting in a Jira case for this, but if you do we'd need a lot more detail: ideally the whole of the metadata file and more detail about how "fails to parse" manifests. There are multiple layers of interpretation being performed so the issue (if any) could be in a number of different places.</div><div><br></div><div><span style="font-size:12px"> -- Ian</span></div><div><span style="border-collapse:separate;font-variant-ligatures:normal;font-variant-east-asian:normal;font-weight:normal;line-height:normal;border-spacing:0px"><div style="word-wrap:break-word"><span style="border-collapse:separate;color:rgb(0,0,0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-east-asian:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;border-spacing:0px"><div><span style="font-size:medium"><br></span></div></span></div></span><br><br>
</div>
<br></div></div>-- <br>
For Consortium Member technical support, see <a href="https://shibboleth.atlassian.net/wiki/x/ZYEpPw" rel="noreferrer" target="_blank">https://shibboleth.atlassian.net/wiki/x/ZYEpPw</a><br>
To unsubscribe from this list send an email to <a href="mailto:users-unsubscribe@shibboleth.net" target="_blank">users-unsubscribe@shibboleth.net</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">Baron Fujimoto <<a href="mailto:baron@hawaii.edu" target="_blank">baron@hawaii.edu</a>> :: UH Information Technology Services<br>minutas cantorum, minutas balorum, minutas carboratum desendus pantorum</div></div>