shibd unable to verify signature when metadata is cached

[Cantor, Scott]

> The only differences an xmldiff reports (once I turned off attribute ordering)
> are that the source document includes a number of 
 encoded
> characters in the  UIInfo extensions. I will have to dig back into the c14n spec
> to remember what that represents (other than the obvious, it's a CR).

Part of c14n is to normalize linefeeds (to LF), and other part is to expand character references.

The interesting thing I wasn't aware of is that Xerces is writing the DOM back out to the file system with CRLF. This happens every time, the file starts with LF only on the server and then gets converted on disk. I would guess that only happens on Windows but I'll have to look into it. It does make me wonder if you're seeing the bug elsewhere.

At least on Windows, what ends up happening is not really "wrong". The extra 
 entities are expanded in the DOM and written back out as literal 0D characters, not illegal (since they'd be converted to 0D internally anyway). So those lines end up with 0D 0D 0A in the output.

If I had to guess at the bug, I'd say Santuario or Xerces (probably the former, during c14n) is collapsing runs of 0D characters instead of normalizing only 0D 0A sequences, and so it loses the extra CRs and the digest fails. I will eyeball the xmlsec code but even if I spotted what seems like a bug, none of that is my code originally and changing c14n code would be tantamount to Russian Roulette. It's so dangerous I'd probably only do it conditionally based on an environment variable or something and leave it off by default just to make it possible to do widespread testing with it on.

Of course in your case the workaround is obvious and should be straightforward.

-- Scott