shibd unable to verify signature when metadata is cached

[Nick Roy]

Hi - the InCommon TAC discussed this issue at a recent call, and it was recommended that I post here to advocate for a resolution to this issue as soon as can be reasonably undertaken.  Since much of the InCommon Service Provider installed base uses Shibboleth SP, this is a critical issue for our deployers.

Thank you,

Nick

From: users <users-bounces at shibboleth.net<mailto:users-bounces at shibboleth.net>> on behalf of Bradley Beddoes <bradleybeddoes at aaf.edu.au<mailto:bradleybeddoes at aaf.edu.au>>
Reply-To: Shib Users <users at shibboleth.net<mailto:users at shibboleth.net>>
Date: Wednesday, March 23, 2016 at 4:36 PM
To: Shib Users <users at shibboleth.net<mailto:users at shibboleth.net>>
Subject: Re: shibd unable to verify signature when metadata is cached

Hi,
Just closing the loop on this one. Removing carriage returns from our internal sources, so they no longer appear when signing the metadata document, has allowed the SP to function correctly, as expected.

I note that an eduGAIN entity also started publishing a description containing `
` within the last few days. This initially broke the eduGAIN signing process but once that was corrected the values eventually flowed out to SP which then had problems.

I'll be adding some further filtering to our raw metadata ingest routines (for sources such as eduGAIN) before we re-publish them as Shibboleth SP is such a critical component for us. This might be a useful consideration for other operators as well, should they have the capability.

cheers,
Bradley

On Tue, Mar 22, 2016 at 1:26 PM, Bradley Beddoes <bradleybeddoes at aaf.edu.au<mailto:bradleybeddoes at aaf.edu.au>> wrote:
Scott,
Thanks for the investigation here.

I've also been able to confirm the `
` characters are the difference between our published document and that persisted by shibboleth SP. oXygen did a much better job than what I had previously in highlighting this, thanks for suggesting it.

I'll go back to our metadata code and filter the extraneous CR before signing to address this issue.

cheers,
Bradley



On Tue, Mar 22, 2016 at 12:36 AM, Cantor, Scott <cantor.2 at osu.edu<mailto:cantor.2 at osu.edu>> wrote:
> Part of c14n is to normalize linefeeds (to LF), and other part is to expand
> character references.

That's actually in the base XML spec. It's XML 1.1, but that hasn't changed, see section 2.11 of [1].

You'll note that single 0D characters have to be converted to 0A. That's what happens here, so 0D 0D 0A ends up as 0A 0A (I'm guessing).

The original source document has entity references in it, so if there's a bug, it may be that Xerces isn't actually converting the 0D that's in the entity reference. It's not immediately clear to me if it should be or not, but whatever it does do is apparently "correct" and consistent with the Java parser.

The round trip to the backup file loses the entity references and at that point the extra 0D characters are clearly linefeed separators and are being normalized out. So they clearly aren't being normalized out during the initial parse. I'm guessing the IdP and Java code create the backup file by copying the bytes directly but I don't know for sure.

I think the basic answer here is that my SP may not allow certain things to be done, more or less as a consequence of the design of the backup creation. It may be that you could use CDATA here and get it to work, but it really depends on the DOM, and I'm not sure that I preserve CDATA nodes either.

It's certainly a bad idea to try and slip in extra CRs whether it's a bug or not.

-- Scott

[1] https://www.w3.org/TR/2004/REC-xml11-20040204/#sec-white-space

--
To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net<mailto:users-unsubscribe at shibboleth.net>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20160412/f31bd30a/attachment.html>