metadata x509 certificate lines terminated with 
 ?

[Baron Fujimoto]

On further investigation, I think I had misdiagnosed the problem. We were
loading the metadata with a typical FileBackedHTTPMetadataProvider. E.g.:

        <MetadataProvider xsi:type="FileBackedHTTPMetadataProvider"
                id="FOO"
                metadataURL="https://foo.example.com/saml/metadata/sp"
                backingFile="%{idp.home}/metadata/foo-metadata.xml" />

However, while poking around, I noticed an existing backingFile that was
empty with size 0. I'm unsure how we came to have this empty backingFile.

The logged error was

ERROR
[org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver:376]
- Metadata Resolver FileBackedHTTPMetadataResolver FOO: Error occurred
while attempting to refresh metadata from '
https://foo.example.com/saml/metadata/sp'
net.shibboleth.utilities.java.support.resolver.ResolverException: Unable to
unmarshall metadata
        at
org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver.unmarshallMetadata(AbstractReloadingMetadataResolver.java:462)
Caused by: org.opensaml.core.xml.io.UnmarshallingException:
net.shibboleth.utilities.java.support.xml.XMLParserException: Unable to
parse inputstream, it contained invalid XML
        at
org.opensaml.saml.metadata.resolver.impl.AbstractMetadataResolver.unmarshallMetadata(AbstractMetadataResolver.java:356)
Caused by: net.shibboleth.utilities.java.support.xml.XMLParserException:
Unable to parse inputstream, it contained invalid XML
        at
net.shibboleth.utilities.java.support.xml.BasicParserPool.parse(BasicParserPool.java:253)
Caused by: org.xml.sax.SAXParseException: Premature end of file.
        at
java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:204)

I'd assumed the "Unable to parse inputstream, it contained invalid XML" was
occuring while trying to load the metadata from the metadataURL, but I
think it was actually occuring while trying to parse the existing empty
backingFile(?), which is consistent with "Premature end of file" (and it
certainly "contained" invalid XML). So I'm guessing the IdP was trying to
check the backingFile contents (to unmarshall it?) before refreshing it
from the metadataURL. After deleting the empty backingFile, the IdP was
able to successfully load the metadata without error from the metadataURL,
and it apparently didn't balk at the "&#xd;" entities in the downloaded
metadata.

Mea culpa.

On Wed, Mar 23, 2022 at 11:57 PM Ian Young <ian at iay.org.uk> wrote:

>
> On 2022-03-24, at 01:59, Jason Pyeron <jpyeron at pdinc.us> wrote:
>
> RFC 7468 allows for whitespace, &#xd; is ASCII 13, a white space char. Why
> not here? The RFC says, in General Considerations:
>
>
> Things are probably a bit more complicated than they appear. When we see
> an encoded CR character like this, either as &#xd; or as 
 it's usually
> because something in the chain got deeply confused about how the end of a
> line should be represented.
>
> On Windows, as you're probably aware, text file lines are CR (12) followed
> by LF (10). On most other systems these days, it's just LF. If something
> gets moved from one kind of system to another, the CR can end up isolated
> from the logical end of line and treated as independent white space.
>
> This sounds harmless (it's just white space, in some sense) but in fact
> can cause issues with things like signatures (on one system, there's just a
> line terminator, on another it's white space then a line terminator).
>
> For example, see https://shibboleth.atlassian.net/browse/SSPCPP-684
>
> I didn't think the IdP was similarly impacted but I guess I'm not
> astonished. As a result of the problems with the SP, several federations
> now look for this and exclude entity-encoded CR characters to avoid issues
> for their members, so we don't see this in the wild any more.
>
> As to this:
>
> "Since we include the "&#xd" characters to show a break in the metadata
> information, we will not be able to remove these characters from the XML
> file"
>
>
> Your supplier is just obviously confused, sorry. The "&#xd;" does nothing
> of the kind. A line break does, and this may once have been part of a line
> break, but it isn't any more.
>
> As you say this just appears in the certificate data, I'd have to say my
> suspicion is that they got bitten by one of the changes that was made to
> Base64 handling in Java a while back, in which things like certificate and
> signature data ends up internally with CR LF sequences inside the Java
> strings; when written out on a non-Windows system, the CRs end up encoded.
> In my opinion, this is nonsensical and a result of an over-reading of the
> Base64 specification but obviously others disagree.
>
> If your supplier thinks this is somehow mandated by the XML standard "to
> show a break in the metadata information", I wonder why they didn't notice
> that the rest of the XML wasn't done the same way.
>
> Having said which, if that's their response I doubt that you'll be able to
> have a real conversation about this. You're probably going to have to
> remove the entities by hand.
>
> The IdP fails to parse this. Is this actually valid SAML for the metadata?
>
>
> It's complicated. It might be worth putting in a Jira case for this, but
> if you do we'd need a lot more detail: ideally the whole of the metadata
> file and more detail about how "fails to parse" manifests. There are
> multiple layers of interpretation being performed so the issue (if any)
> could be in a number of different places.
>
>     -- Ian
>
>
>
>
> --
> For Consortium Member technical support, see
> https://shibboleth.atlassian.net/wiki/x/ZYEpPw
> To unsubscribe from this list send an email to
> users-unsubscribe at shibboleth.net
>


-- 
Baron Fujimoto <baron at hawaii.edu> :: UH Information Technology Services
minutas cantorum, minutas balorum, minutas carboratum desendus pantorum
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20220324/0236e88e/attachment.htm>